-- Logs begin at Fri 2022-08-26 08:10:05 UTC, end at Fri 2022-08-26 09:14:45 UTC. -- Aug 26 08:36:03.437483 dv-cinder-852364-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 26 08:36:03.447103 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 26 08:36:03.449148 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 26 08:36:03 2022] *** Aug 26 08:36:03.449148 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 26 08:36:03.449148 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 26 08:36:03.449148 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: nodename: dv-cinder-852364-5-iscsi Aug 26 08:36:03.449148 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: machine: x86_64 Aug 26 08:36:03.449148 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: clock source: unix Aug 26 08:36:03.449148 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: pcre jit disabled Aug 26 08:36:03.449148 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: detected number of CPU cores: 4 Aug 26 08:36:03.449148 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: current working directory: / Aug 26 08:36:03.449148 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: detected binary path: /usr/bin/uwsgi-core Aug 26 08:36:03.449148 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: your processes number limit is 47680 Aug 26 08:36:03.449148 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: your memory page size is 4096 bytes Aug 26 08:36:03.449148 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: detected max file descriptor number: 2048 Aug 26 08:36:03.449148 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: lock engine: pthread robust mutexes Aug 26 08:36:03.450356 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: thunder lock: enabled Aug 26 08:36:03.450356 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 26 08:36:03.450356 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 26 08:36:03.490214 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: Python main interpreter initialized at 0x55a60e84f9c0 Aug 26 08:36:03.490214 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: python threads support enabled Aug 26 08:36:03.490214 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: your server socket listen backlog is limited to 100 connections Aug 26 08:36:03.490214 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: your mercy for graceful operations on workers is 90 seconds Aug 26 08:36:03.490614 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: mapped 403077 bytes (393 KB) for 2 cores Aug 26 08:36:03.490614 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: *** Operational MODE: preforking *** Aug 26 08:36:03.490973 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: *** uWSGI is running in multiple interpreter mode *** Aug 26 08:36:03.490973 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: spawned uWSGI master process (pid: 93164) Aug 26 08:36:03.491207 dv-cinder-852364-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 26 08:36:03.494920 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: spawned uWSGI worker 1 (pid: 93172, cores: 1) Aug 26 08:36:03.494920 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: spawned uWSGI worker 2 (pid: 93173, cores: 1) Aug 26 08:36:03.494920 dv-cinder-852364-5-iscsi devstack@keystone.service[93164]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 26 08:36:07.996714 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 08:36:07.996714 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: from cryptography.utils import int_from_bytes Aug 26 08:36:07.997551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 08:36:07.997551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: from cryptography.utils import int_from_bytes Aug 26 08:36:08.049230 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 26 08:36:08.049516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 26 08:36:08.049627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] command line args: [] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 26 08:36:08.049841 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 26 08:36:08.050015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ================================================================================ {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 26 08:36:08.050274 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] admin_token = **** {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.050512 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] config_dir = [] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.050699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.050912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] config_source = [] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.051099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] debug = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.051301 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.051672 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.051851 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.052107 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.052357 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.052590 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] list_limit = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.052760 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] log_config_append = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.052938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.053120 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] log_dir = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.053286 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] log_file = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.053519 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.053763 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.054005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.054283 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.054509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.054742 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.054916 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.055088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.055300 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.055529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.055756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.055985 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.056243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.056480 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.056757 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.057274 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.71/identity {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.057504 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] publish_errors = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.057673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.057894 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.058096 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.058324 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.058538 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.058768 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] standard_threads = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.059031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.059269 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.059443 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.059636 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] use_journal = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.059829 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] use_json = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.059957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] use_stderr = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.060129 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] use_syslog = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.060359 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.060617 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.060853 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.061111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.061348 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.061594 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.061889 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.062140 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.062378 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.062624 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.062869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.063145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.063379 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.063624 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.063855 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.064099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.064678 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.064919 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.065178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.065434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.065680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.065914 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.066165 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.066429 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.066679 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.066964 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.067184 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.067414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.067644 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.067875 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.068124 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.068371 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.068597 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.068825 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.069068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.069315 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.069548 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.069794 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.070025 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.070257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.070490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.070718 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.070969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.071199 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.071426 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.071658 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.071903 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.072137 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.072378 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.072581 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.072826 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.073051 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.073295 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.073530 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.073763 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.073996 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.074229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.074461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.074692 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.074972 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.075199 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.075892 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.076137 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.076374 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.076643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.076878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.077113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.077352 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.077645 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.077918 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.078155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.078403 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.078653 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.078911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.079148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.079384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.079628 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.079868 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.080102 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.080346 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.080582 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.080827 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.081063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.081297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.081530 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.081759 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.081993 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.082256 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.082494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.082737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.083007 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.083246 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.083488 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.083724 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.083959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.084192 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.084421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.084650 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.084878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.085109 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.085336 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.085575 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.085807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.086053 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.086287 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.086540 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.086800 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.087069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.087322 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] auth.external = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.087530 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.087828 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.088058 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.088286 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] auth.password = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.088523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] auth.token = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.088755 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.088988 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.089238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.089460 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.089707 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.089957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.090183 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.090415 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] credential.caching = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.090662 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.090926 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.091175 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.091413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.091661 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.091890 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.092134 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.092382 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.092621 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.092870 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.093227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.093481 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.093741 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.093989 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.094243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.094474 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] federation.caching = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.094733 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.094975 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.095222 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.095452 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.095696 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.095930 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.096178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.096411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.096663 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.096895 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.097138 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.097368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.caching = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.097611 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.097855 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.098089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.098321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.098563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.098790 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.099047 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.099294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.099545 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.099774 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.100002 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.100261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.100477 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.100677 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.100952 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.101158 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.101412 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.101679 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.101913 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.102154 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.102373 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.102597 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.102856 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.103091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.103336 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.103574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.103823 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.104149 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.104379 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.104625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.104862 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.105557 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.105814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.106053 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.106294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.106693 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.106968 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.107203 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.107456 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.107698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.107940 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.108195 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.108452 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.108748 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.109023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.109275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.109524 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.109761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.110000 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.110237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.110469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.110720 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.111319 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.111557 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.111802 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.112049 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.112298 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.112537 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.112772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.113020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.113261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.113493 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.113731 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.113971 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.114216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.114459 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.114704 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.114979 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.115211 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.115467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.115700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.115936 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.116168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.116407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.116642 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.116902 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.117142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.117394 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.117623 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.117863 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.118101 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.118340 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.118574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.118818 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.119089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.119323 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.119565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.119798 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] resource.caching = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.120053 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.120306 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.120543 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.120793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.121041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.121285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.121507 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.121740 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.121978 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.122215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] role.caching = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.122465 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] role.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.122700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.122981 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.123229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.123472 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.123714 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.123960 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.124202 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.124527 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.124778 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.125009 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.125261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.125498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.125743 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.126004 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.126468 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.126702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.126991 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.127251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.127507 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.127753 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.127986 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.128241 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.128501 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.128747 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.128969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.129197 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.129429 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.129682 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.129946 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.130194 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.130427 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.130664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.130906 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.131140 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] token.caching = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.131375 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.131627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.131869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.132120 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.132359 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.132596 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.132838 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.133082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.133327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.133565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.133801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.134034 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.134278 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.134557 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.134784 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.135057 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.135324 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.135586 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.connection = **** {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.135828 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.136072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.136728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.137018 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.137251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.137669 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.138083 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.138199 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.138384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.138626 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.138868 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.139134 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.139389 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.139625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.139866 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.140177 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.140483 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.140720 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.140892 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93172) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 26 08:36:08.176271 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.176531 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.176699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.186105 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.217111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.217359 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.279314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.338178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.338470 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.351195 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 08:36:08.351195 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: from cryptography.utils import int_from_bytes Aug 26 08:36:08.352195 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 08:36:08.352195 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: from cryptography.utils import int_from_bytes Aug 26 08:36:08.404342 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 26 08:36:08.404600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 26 08:36:08.404751 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] command line args: [] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 26 08:36:08.404969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 26 08:36:08.405134 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ================================================================================ {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 26 08:36:08.405388 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] admin_token = **** {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.405670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] config_dir = [] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.405872 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.406057 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] config_source = [] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.406238 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] debug = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.406498 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.406666 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.406842 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.407139 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.407392 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.407625 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] list_limit = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.407806 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] log_config_append = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.407986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.408179 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] log_dir = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.408357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] log_file = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.408551 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.408801 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.409046 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.409318 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.409483 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.409650 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.409898 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.410150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.410390 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.410693 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.410971 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.411213 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.411470 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.411723 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.412011 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.412632 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.71/identity {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.412812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] publish_errors = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.413053 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.413289 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.413468 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.413722 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.413963 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.414270 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] standard_threads = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.414509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.414755 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.414958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.415141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] use_journal = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.415320 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] use_json = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.415504 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] use_stderr = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.415679 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] use_syslog = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.415920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 08:36:08.416186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.416431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.416700 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.416942 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.417196 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.417439 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.417690 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.417932 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.418185 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.418432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.418690 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.418960 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.419215 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.419452 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.419702 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.419953 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.420190 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.420447 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.420708 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.420956 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.421225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.421517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.421787 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.422083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.422343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.422580 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.422816 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.423080 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.423321 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.423576 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.423825 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.424060 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.424294 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.424545 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.424796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.425037 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.425288 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.425525 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.425763 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.426002 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.426236 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.426469 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.426705 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.426980 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.427220 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.427474 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.427708 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.427950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.428184 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.428422 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.428659 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.428912 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.429145 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.429386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.429625 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.429861 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.430100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.430340 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.430575 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.430832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.431102 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.431347 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.431585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.431854 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.432091 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.432476 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.432686 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.432816 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.433067 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.433321 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.433556 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.433815 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.434057 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.434295 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.434533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.434785 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.435055 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.435294 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.435542 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.435782 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.436034 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.436275 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.436511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.436748 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.437026 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.437265 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.437539 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.437785 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.438037 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.438285 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.438528 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.438776 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.439048 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.439290 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.439530 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.439765 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.439997 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.440232 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.440475 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.440708 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.441185 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.441185 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.441423 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.441652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.441986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.442344 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.442596 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.442833 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] auth.external = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.443099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.443407 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.443644 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.443899 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] auth.password = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.444115 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] auth.token = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.444361 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.444599 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.444850 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.445087 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.445338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.445592 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.445827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.446231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] credential.caching = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.446971 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.447158 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.447624 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.448090 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.448511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.448939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.449416 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.449889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.450327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.450654 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.450997 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.451269 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.451551 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.451808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.452084 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.452307 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] federation.caching = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.452635 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.452894 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.453150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.453421 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.453631 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.453989 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.454196 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.454362 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.454628 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.454866 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.455148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.455391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.caching = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.455646 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.455903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.456144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.456387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.456642 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.456879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.457124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.457380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.457640 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.457878 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.458115 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.458354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.458604 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.458861 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.459220 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.459543 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.459745 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.459971 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.460219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.460465 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.460704 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.460960 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.461181 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.461424 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.461675 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.461923 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.462176 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.462414 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.462664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.462945 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.463191 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.463442 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.463691 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.463924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.464170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.464404 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.464645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.464884 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.465135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.465377 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.465616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.465872 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.466124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.466358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.466595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.466848 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.467126 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.467366 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.467605 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.467845 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.468122 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.468384 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.468813 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.469049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.469332 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.469557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.469812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.470050 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.470287 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.470533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.470774 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.471045 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.471283 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.471563 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.471831 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.472077 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.472331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.472579 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.472813 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.473071 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.473311 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.473549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.473789 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.474026 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.474278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.474528 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.474788 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.475051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.475289 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.475613 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.475826 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.475998 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.476240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.476490 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.476736 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.476973 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.477213 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.477456 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] resource.caching = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.477712 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.477962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.478201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.478450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.478704 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.478973 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.479224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.479462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.479704 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.479942 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] role.caching = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.480193 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] role.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.480428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.480685 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.480972 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.481165 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.481424 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.481670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.481921 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.482170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.482424 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.482653 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.482920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.483178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.483426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.483697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.484173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.484414 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.484669 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.484918 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.485168 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.485422 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.485658 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.486239 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.486490 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.486733 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.487006 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.487243 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.487477 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.487735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.487997 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.488252 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.488493 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.488731 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.488966 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.489204 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] token.caching = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.489448 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.489699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.489940 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.490198 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.490447 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.490692 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.490971 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.491220 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.491562 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.491935 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.492188 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.492427 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.492677 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.492979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.493212 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.493458 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.493728 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.493996 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.connection = **** {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.494246 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.494493 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.494734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.495010 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.495253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.495491 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.495727 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.495965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.496214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.496454 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.496700 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.496938 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.497192 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.497505 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.497767 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.498002 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.498247 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.498485 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 08:36:08.498682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93173) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 26 08:36:08.525746 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.526006 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.526190 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.532348 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.532641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.533020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.533387 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.533771 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.534136 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.535518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.555987 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.556305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.574451 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.574765 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.575368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.575690 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.576007 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.576358 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.576637 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.597787 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.598268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.619865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.620197 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.620535 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.620867 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.621202 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.621541 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.621902 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93172) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.636825 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.637092 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.640670 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.640939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 08:36:08.641224 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.641424 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 08:36:08.641703 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.642161 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.643111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.643111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.653114 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.653417 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.653688 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.653970 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.661193 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.664261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.664556 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.664843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.665128 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.665413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.665677 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.665950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.666227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.666643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.666948 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.667228 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.690348 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.692015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.692313 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.698156 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.698473 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.698757 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.706189 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.706463 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.711391 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.711708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.711995 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.712282 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.712563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.712842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.713117 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.713655 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.732719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.733016 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.737580 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.741120 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.741435 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.745877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.750070 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.753579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.753901 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.754190 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.754474 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.754750 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.755090 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.772582 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.772933 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.773248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.773520 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.775412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.775646 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.775873 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.776099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.776325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.776546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.781144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.782759 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.783007 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.785669 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.785973 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.786272 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.786555 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.786832 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.790274 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.790511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.790737 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.790990 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.791219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.791435 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.791650 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.798460 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.798765 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.799091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.799358 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.799645 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.799923 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.800104 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.800256 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.803522 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.803736 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.803956 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.804177 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.804391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.804605 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.804820 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93173) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 08:36:08.815442 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.815697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 08:36:08.815994 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.816373 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 08:36:08.816596 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.816768 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.816768 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.817000 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.817235 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.819948 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.822877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.824361 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.825820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.826108 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.826391 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.826977 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.827298 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.827598 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.827889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.834055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.835329 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.835750 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.838335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.838627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.838782 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.838936 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.839099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.839245 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.839512 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.839676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.839802 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.839938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.840077 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.840325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.840624 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.840902 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.841200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.848291 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.848600 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.848887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.855953 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93172) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.856265 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.856555 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.856824 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.857096 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.857377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.857648 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.857937 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.858219 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.858495 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.858843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.859157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.859436 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.859847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93172) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.864892 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.866426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.866714 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.871407 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.871711 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.872000 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.879031 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.879367 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.883513 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.883835 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.884127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.884433 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.884730 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.885135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.885420 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.885714 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.895702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55a60e84f9c0 pid: 93172 (default app) Aug 26 08:36:08.899902 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128efca-bc7e-4105-970d-c9c4c5cc028d None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:08.900198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128efca-bc7e-4105-970d-c9c4c5cc028d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:08.900455 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128efca-bc7e-4105-970d-c9c4c5cc028d None None] PATH_INFO: `/v3` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:08.903483 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.903796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.904975 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1/1] 10.222.0.71 () {50 vars in 791 bytes} [Fri Aug 26 08:36:08 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:08.907923 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.911309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.911624 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.916146 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.919823 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.922707 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.923047 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.923396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.923771 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.924062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.924364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.940873 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.941188 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.941467 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.941737 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.948791 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.950546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.950834 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.953617 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.953924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.954213 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.954500 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.954794 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.966266 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.966580 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.967036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.967184 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.967424 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.967723 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.968005 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.983884 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.987378 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.990491 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.991993 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.993527 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:08.993797 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:08.994085 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:09.000746 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:09.002273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:09.005334 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:09.005746 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:09.006057 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:09.006338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:09.014473 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:09.014844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:09.015174 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:09.022291 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93173) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 08:36:09.022730 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:09.023053 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:09.023340 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:09.023640 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:09.023924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:09.024208 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:09.024503 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:09.024785 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:09.025170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:09.025501 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:09.025775 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:09.026202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:09.026346 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93173) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 08:36:09.082288 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55a60e84f9c0 pid: 93173 (default app) Aug 26 08:36:16.633672 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c458e-a34a-43a6-8bee-41451eb38087 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:16.634216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c458e-a34a-43a6-8bee-41451eb38087 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:16.634216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c458e-a34a-43a6-8bee-41451eb38087 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:16.635579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 2/2] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:36:16.646049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fb3ac-fe88-4d56-b38e-9ff7ad69506c None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:16.646963 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fb3ac-fe88-4d56-b38e-9ff7ad69506c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:16.647110 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fb3ac-fe88-4d56-b38e-9ff7ad69506c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:16.752853 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: INFO dbcounter [None req-704fb3ac-fe88-4d56-b38e-9ff7ad69506c None None] Registered counter for database keystone Aug 26 08:36:16.777659 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG oslo_db.sqlalchemy.engines [None req-704fb3ac-fe88-4d56-b38e-9ff7ad69506c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93173) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 26 08:36:16.784950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writer thread running {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 26 08:36:16.993604 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG passlib.handlers.bcrypt [None req-704fb3ac-fe88-4d56-b38e-9ff7ad69506c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=93173) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 26 08:36:16.993955 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG passlib.handlers.bcrypt [None req-704fb3ac-fe88-4d56-b38e-9ff7ad69506c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93173) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 26 08:36:17.051991 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-704fb3ac-fe88-4d56-b38e-9ff7ad69506c None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:17.053645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-704fb3ac-fe88-4d56-b38e-9ff7ad69506c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:17.218353 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1/3] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 577 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 26 08:36:17.815132 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c458e-a34a-43a6-8bee-41451eb38087 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:17.909243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: INFO dbcounter [None req-3a1c458e-a34a-43a6-8bee-41451eb38087 None None] Registered counter for database keystone Aug 26 08:36:17.933023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG oslo_db.sqlalchemy.engines [None req-3a1c458e-a34a-43a6-8bee-41451eb38087 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93172) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 26 08:36:17.940688 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writer thread running {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 26 08:36:18.063364 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c458e-a34a-43a6-8bee-41451eb38087 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:18.068194 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc617d2-0188-4572-b0d0-df8481fe114f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:18.068822 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc617d2-0188-4572-b0d0-df8481fe114f admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:18.069172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc617d2-0188-4572-b0d0-df8481fe114f admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:18.069443 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc617d2-0188-4572-b0d0-df8481fe114f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:18.079939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc617d2-0188-4572-b0d0-df8481fe114f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:18.140791 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: WARNING keystone.common.rbac_enforcer.enforcer [None req-dbc617d2-0188-4572-b0d0-df8481fe114f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 26 08:36:18.141148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc617d2-0188-4572-b0d0-df8481fe114f admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:18.154565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 3/4] 10.222.0.71 () {58 vars in 1324 bytes} [Fri Aug 26 08:36:17 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 08:36:18.162605 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704fb3ac-fe88-4d56-b38e-9ff7ad69506c None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:18.177799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704fb3ac-fe88-4d56-b38e-9ff7ad69506c None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:18.182380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30af79f-ad60-4eb5-9350-ae96dee61997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:18.183568 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30af79f-ad60-4eb5-9350-ae96dee61997 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:18.183852 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30af79f-ad60-4eb5-9350-ae96dee61997 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:18.184127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30af79f-ad60-4eb5-9350-ae96dee61997 admin admin] PATH_INFO: `/v3/projects/45132fb08c454ebaa1180c0b933c68b7` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:18.185647 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30af79f-ad60-4eb5-9350-ae96dee61997 admin admin] RBAC: Authorizing `identity:get_project(project_id=45132fb08c454ebaa1180c0b933c68b7)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:18.241983 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f30af79f-ad60-4eb5-9350-ae96dee61997 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 26 08:36:18.242301 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30af79f-ad60-4eb5-9350-ae96dee61997 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:18.247193 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 2/5] 10.222.0.71 () {56 vars in 1207 bytes} [Fri Aug 26 08:36:18 2022] GET /identity/v3/projects/45132fb08c454ebaa1180c0b933c68b7 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:20.216826 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb213956-482a-4955-92c4-9fd86bfbabe1 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:20.217405 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb213956-482a-4955-92c4-9fd86bfbabe1 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:20.217405 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb213956-482a-4955-92c4-9fd86bfbabe1 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:20.221737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 4/6] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:36:20.229199 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438098de-8ef8-4680-9ec3-23f4528f6068 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:20.229768 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438098de-8ef8-4680-9ec3-23f4528f6068 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:20.230175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438098de-8ef8-4680-9ec3-23f4528f6068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:20.257844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-438098de-8ef8-4680-9ec3-23f4528f6068 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:20.259491 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-438098de-8ef8-4680-9ec3-23f4528f6068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:20.271748 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 3/7] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:20 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 26 08:36:20.538200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb213956-482a-4955-92c4-9fd86bfbabe1 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:20.552078 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb213956-482a-4955-92c4-9fd86bfbabe1 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:20.557425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab5ab5-c6a3-4f3a-9615-6ca84b52536d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:20.559216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab5ab5-c6a3-4f3a-9615-6ca84b52536d admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:20.559216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab5ab5-c6a3-4f3a-9615-6ca84b52536d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:20.559216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab5ab5-c6a3-4f3a-9615-6ca84b52536d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:20.569682 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ab5ab5-c6a3-4f3a-9615-6ca84b52536d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:20.571855 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ab5ab5-c6a3-4f3a-9615-6ca84b52536d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:20.583287 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 5/8] 10.222.0.71 () {58 vars in 1324 bytes} [Fri Aug 26 08:36:20 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 08:36:20.589401 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438098de-8ef8-4680-9ec3-23f4528f6068 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:20.599731 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438098de-8ef8-4680-9ec3-23f4528f6068 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:20.604669 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a48b79-c6d5-4260-8390-57bb6c15e3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:20.605354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a48b79-c6d5-4260-8390-57bb6c15e3af admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:20.605639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a48b79-c6d5-4260-8390-57bb6c15e3af admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:20.605915 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a48b79-c6d5-4260-8390-57bb6c15e3af admin admin] PATH_INFO: `/v3/users/668b82af2c1345cca572ef92fff86ff2` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:20.607533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a48b79-c6d5-4260-8390-57bb6c15e3af admin admin] RBAC: Authorizing `identity:get_user(user_id=668b82af2c1345cca572ef92fff86ff2)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:20.609492 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a48b79-c6d5-4260-8390-57bb6c15e3af admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:20.611253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 4/9] 10.222.0.71 () {56 vars in 1198 bytes} [Fri Aug 26 08:36:20 2022] GET /identity/v3/users/668b82af2c1345cca572ef92fff86ff2 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:22.698283 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263298f9-b141-4a23-a92a-3cacdea6420c None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:22.699127 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263298f9-b141-4a23-a92a-3cacdea6420c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:22.699127 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263298f9-b141-4a23-a92a-3cacdea6420c None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:22.700271 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 6/10] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:22.708497 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44959e0c-d411-4f16-be57-b964ae03a9e2 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:22.709176 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44959e0c-d411-4f16-be57-b964ae03a9e2 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:22.709176 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44959e0c-d411-4f16-be57-b964ae03a9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:22.734684 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-44959e0c-d411-4f16-be57-b964ae03a9e2 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:22.736223 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-44959e0c-d411-4f16-be57-b964ae03a9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:22.789465 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08753e-d54e-4a1f-8687-5f21ce552641 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:22.790067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08753e-d54e-4a1f-8687-5f21ce552641 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:22.790629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08753e-d54e-4a1f-8687-5f21ce552641 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:22.793340 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 7/11] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:36:22.802558 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47237053-cec8-498a-8cd2-634b64c53e54 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:22.803092 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47237053-cec8-498a-8cd2-634b64c53e54 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:22.803588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47237053-cec8-498a-8cd2-634b64c53e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:22.831358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 5/12] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:36:22.873244 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG passlib.handlers.bcrypt [None req-47237053-cec8-498a-8cd2-634b64c53e54 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=93172) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 26 08:36:22.873775 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG passlib.handlers.bcrypt [None req-47237053-cec8-498a-8cd2-634b64c53e54 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93172) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 26 08:36:22.920155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-47237053-cec8-498a-8cd2-634b64c53e54 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:22.921974 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-47237053-cec8-498a-8cd2-634b64c53e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:23.039319 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 8/13] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:36:23.098769 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44959e0c-d411-4f16-be57-b964ae03a9e2 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:23.126189 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:23.228181 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44959e0c-d411-4f16-be57-b964ae03a9e2 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:23.284961 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0a476-b15c-4ee7-8c2e-364c960d94a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:23.285623 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0a476-b15c-4ee7-8c2e-364c960d94a0 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:23.285903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0a476-b15c-4ee7-8c2e-364c960d94a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:23.286174 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0a476-b15c-4ee7-8c2e-364c960d94a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:23.300819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0a476-b15c-4ee7-8c2e-364c960d94a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:36:23.312391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-3eb0a476-b15c-4ee7-8c2e-364c960d94a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:36:23.312391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:36:23.312391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:36:23.312391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:36:23.312391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:36:23.312391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:36:23.312391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:36:23.312391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:36:23.312391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:36:23.312391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:36:23.312391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:36:23.312391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:36:23.312391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:36:23.312391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:36:23.312391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:36:23.312391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:36:23.314309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:36:23.314309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:36:23.314309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:36:23.314309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:36:23.314309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:36:23.314309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:36:23.314309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:36:23.314309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:36:23.314309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:36:23.314309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:36:23.314309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:36:23.314309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:36:23.314309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:36:23.314309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:36:23.314309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:36:23.314309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:36:23.315240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:36:23.315240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:36:23.315240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:36:23.315240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:36:23.315240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:36:23.315240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:36:23.315240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:36:23.315240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:36:23.315240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:36:23.315240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:36:23.315240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:36:23.315240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:36:23.315240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 6/14] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:36:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:36:23.322533 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47237053-cec8-498a-8cd2-634b64c53e54 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:23.347484 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0a476-b15c-4ee7-8c2e-364c960d94a0 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:23.470536 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47237053-cec8-498a-8cd2-634b64c53e54 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:23.474250 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:23.494694 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:23.518978 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0a476-b15c-4ee7-8c2e-364c960d94a0 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:23.544359 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302879ca-6ed9-42e0-b6e4-62ee8d13c6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:23.545741 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302879ca-6ed9-42e0-b6e4-62ee8d13c6ed None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:23.546136 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302879ca-6ed9-42e0-b6e4-62ee8d13c6ed None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:23.546419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302879ca-6ed9-42e0-b6e4-62ee8d13c6ed None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:23.547986 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302879ca-6ed9-42e0-b6e4-62ee8d13c6ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:23.548408 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302879ca-6ed9-42e0-b6e4-62ee8d13c6ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:23.550503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302879ca-6ed9-42e0-b6e4-62ee8d13c6ed None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:23.569085 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d947a3-6ab6-42aa-9720-7f7fa530c5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:23.569648 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d947a3-6ab6-42aa-9720-7f7fa530c5cc None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:23.570051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d947a3-6ab6-42aa-9720-7f7fa530c5cc None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:23.570474 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d947a3-6ab6-42aa-9720-7f7fa530c5cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:23.571304 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 9/15] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:36:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:36:23.579887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302879ca-6ed9-42e0-b6e4-62ee8d13c6ed None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:23.682150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d947a3-6ab6-42aa-9720-7f7fa530c5cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:23.684462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d947a3-6ab6-42aa-9720-7f7fa530c5cc None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:23.700321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302879ca-6ed9-42e0-b6e4-62ee8d13c6ed None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:23.747329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf6d43-5441-492a-a022-0e6c35842fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:23.748226 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf6d43-5441-492a-a022-0e6c35842fe5 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:23.748506 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf6d43-5441-492a-a022-0e6c35842fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:23.748799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf6d43-5441-492a-a022-0e6c35842fe5 None admin] PATH_INFO: `/v3/users/668b82af2c1345cca572ef92fff86ff2` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:23.750724 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bf6d43-5441-492a-a022-0e6c35842fe5 None admin] RBAC: Authorizing `identity:get_user(user_id=668b82af2c1345cca572ef92fff86ff2)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:23.752407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bf6d43-5441-492a-a022-0e6c35842fe5 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:23.754484 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 10/16] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:36:23 2022] GET /identity/v3/users/668b82af2c1345cca572ef92fff86ff2 => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:36:23.760705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf6d43-5441-492a-a022-0e6c35842fe5 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:23.811556 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:23.813522 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:23.837542 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 7/17] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:36:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:36:23.844040 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d947a3-6ab6-42aa-9720-7f7fa530c5cc None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:23.920752 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf6d43-5441-492a-a022-0e6c35842fe5 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:23.981161 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7ad9f-952a-41ef-95fe-89b327f56a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:23.981768 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d947a3-6ab6-42aa-9720-7f7fa530c5cc None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:23.982252 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7ad9f-952a-41ef-95fe-89b327f56a17 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:23.982755 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7ad9f-952a-41ef-95fe-89b327f56a17 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:23.983284 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7ad9f-952a-41ef-95fe-89b327f56a17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:23.985437 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7ad9f-952a-41ef-95fe-89b327f56a17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:23.988498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7ad9f-952a-41ef-95fe-89b327f56a17 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:23.991674 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 11/18] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:36:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:36:24.001540 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7ad9f-952a-41ef-95fe-89b327f56a17 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:24.055887 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b03b0-70f0-4f0b-aa98-66e92011cc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:24.056984 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b03b0-70f0-4f0b-aa98-66e92011cc64 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:24.057322 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b03b0-70f0-4f0b-aa98-66e92011cc64 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:24.057789 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b03b0-70f0-4f0b-aa98-66e92011cc64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:24.078021 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b03b0-70f0-4f0b-aa98-66e92011cc64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:24.081187 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b03b0-70f0-4f0b-aa98-66e92011cc64 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:24.108627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-153b03b0-70f0-4f0b-aa98-66e92011cc64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:36:24.108627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:36:24.108627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:36:24.108627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:36:24.108627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:36:24.108627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:36:24.108627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:36:24.108627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:36:24.108627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:36:24.108627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:36:24.108627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:36:24.108627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:36:24.108627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:36:24.108627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:36:24.108627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:36:24.108627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:36:24.110781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:36:24.110781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:36:24.110781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:36:24.110781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:36:24.110781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:36:24.110781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:36:24.110781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:36:24.110781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:36:24.110781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:36:24.110781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:36:24.110781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:36:24.110781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:36:24.110781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:36:24.110781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:36:24.110781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:36:24.110781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:36:24.110781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:36:24.113010 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:36:24.113010 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:36:24.113010 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:36:24.115131 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 8/19] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:36:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:36:24.117488 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b03b0-70f0-4f0b-aa98-66e92011cc64 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:24.152111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7ad9f-952a-41ef-95fe-89b327f56a17 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:24.154257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:24.175915 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:24.208266 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f50285-aacb-4126-9eec-9db98d77a1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:24.209263 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f50285-aacb-4126-9eec-9db98d77a1a0 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:24.209716 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f50285-aacb-4126-9eec-9db98d77a1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:24.210189 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f50285-aacb-4126-9eec-9db98d77a1a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:24.211556 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f50285-aacb-4126-9eec-9db98d77a1a0 None admin] RBAC: Adding query filter params (role.id=18c3c8a821b647f7982a955880bd0a9a, scope.domain.id=default, user.id=668b82af2c1345cca572ef92fff86ff2) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:24.212092 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f50285-aacb-4126-9eec-9db98d77a1a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:24.214769 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f50285-aacb-4126-9eec-9db98d77a1a0 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:24.229459 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 12/20] 10.222.0.71 () {56 vars in 1322 bytes} [Fri Aug 26 08:36:24 2022] GET /identity/v3/role_assignments?user.id=668b82af2c1345cca572ef92fff86ff2&scope.domain.id=default&role.id=18c3c8a821b647f7982a955880bd0a9a => generated 233 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:36:24.283336 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b03b0-70f0-4f0b-aa98-66e92011cc64 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:24.339350 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7642fb-66a9-44c2-8c6f-b179d6ecfac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:24.340094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7642fb-66a9-44c2-8c6f-b179d6ecfac1 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:24.340374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7642fb-66a9-44c2-8c6f-b179d6ecfac1 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:24.340619 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7642fb-66a9-44c2-8c6f-b179d6ecfac1 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:24.341329 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7642fb-66a9-44c2-8c6f-b179d6ecfac1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:24.341629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7642fb-66a9-44c2-8c6f-b179d6ecfac1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:24.343207 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7642fb-66a9-44c2-8c6f-b179d6ecfac1 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:24.373349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 9/21] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:36:27.013946 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59640d-28b0-40a1-8333-89008ef73194 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:27.014519 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59640d-28b0-40a1-8333-89008ef73194 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:27.014813 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59640d-28b0-40a1-8333-89008ef73194 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:27.017686 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 13/22] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:27.025017 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298d45b-72f7-4afe-bfe3-e8481b25066d None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:27.025582 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298d45b-72f7-4afe-bfe3-e8481b25066d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:27.025906 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298d45b-72f7-4afe-bfe3-e8481b25066d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:27.063151 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-2298d45b-72f7-4afe-bfe3-e8481b25066d None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:27.065148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-2298d45b-72f7-4afe-bfe3-e8481b25066d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:27.192598 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fa200-2da6-4e61-9bd5-7bca01aa7eb5 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:27.193023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fa200-2da6-4e61-9bd5-7bca01aa7eb5 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:27.193447 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fa200-2da6-4e61-9bd5-7bca01aa7eb5 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:27.196190 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 14/23] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:27.201639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fdf98-4334-4351-bca6-f97ab4cac957 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:27.201944 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fdf98-4334-4351-bca6-f97ab4cac957 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:27.202193 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fdf98-4334-4351-bca6-f97ab4cac957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:27.254239 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-6e5fdf98-4334-4351-bca6-f97ab4cac957 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:27.256349 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-6e5fdf98-4334-4351-bca6-f97ab4cac957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:27.263074 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 10/24] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:36:27.438800 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 15/25] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 08:36:27.683477 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ab2fe-1220-4fe8-9378-798c62007301 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:27.683965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ab2fe-1220-4fe8-9378-798c62007301 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:27.684448 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ab2fe-1220-4fe8-9378-798c62007301 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:27.687328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 11/26] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:27.695258 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cb48d-9c22-4087-a040-448586ea1559 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:27.695747 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cb48d-9c22-4087-a040-448586ea1559 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:27.696228 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cb48d-9c22-4087-a040-448586ea1559 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:27.700055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c1d72-7f74-4abd-930a-fce8e1f8b815 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:27.700902 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 12/27] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:27.701104 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c1d72-7f74-4abd-930a-fce8e1f8b815 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:27.701597 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c1d72-7f74-4abd-930a-fce8e1f8b815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:27.713408 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfea9d0-d2d6-46b8-952e-a2f247fcaf65 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:27.713896 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfea9d0-d2d6-46b8-952e-a2f247fcaf65 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:27.714369 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfea9d0-d2d6-46b8-952e-a2f247fcaf65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:27.739633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-585c1d72-7f74-4abd-930a-fce8e1f8b815 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:27.741229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-585c1d72-7f74-4abd-930a-fce8e1f8b815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:27.751261 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-1cfea9d0-d2d6-46b8-952e-a2f247fcaf65 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:27.753277 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1cfea9d0-d2d6-46b8-952e-a2f247fcaf65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:27.798913 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:27.899742 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 13/28] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:36:27.900629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfea9d0-d2d6-46b8-952e-a2f247fcaf65 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:27.904226 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 16/29] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:36:27.905403 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c1d72-7f74-4abd-930a-fce8e1f8b815 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:27.970146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:28.114685 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c1d72-7f74-4abd-930a-fce8e1f8b815 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:28.135972 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfea9d0-d2d6-46b8-952e-a2f247fcaf65 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:28.190584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d25714-f658-4537-b7da-e6b035c204b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:28.192473 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d25714-f658-4537-b7da-e6b035c204b3 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:28.193207 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d25714-f658-4537-b7da-e6b035c204b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:28.195087 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d25714-f658-4537-b7da-e6b035c204b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:28.216916 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca2d67-d19b-42c1-8e59-75093a12bb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:28.217957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca2d67-d19b-42c1-8e59-75093a12bb6d None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:28.218432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca2d67-d19b-42c1-8e59-75093a12bb6d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:28.219084 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca2d67-d19b-42c1-8e59-75093a12bb6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:28.238564 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ca2d67-d19b-42c1-8e59-75093a12bb6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:28.239531 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d25714-f658-4537-b7da-e6b035c204b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:36:28.240705 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ca2d67-d19b-42c1-8e59-75093a12bb6d None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:28.248552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-13d25714-f658-4537-b7da-e6b035c204b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:36:28.248552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:36:28.248552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:36:28.248552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:36:28.248552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:36:28.248552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:36:28.248552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:36:28.248552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:36:28.248552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:36:28.248552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:36:28.248552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:36:28.248552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:36:28.248552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:36:28.248552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:36:28.248552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:36:28.248552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:36:28.249489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:36:28.249489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:36:28.249489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:36:28.249489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:36:28.249489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:36:28.249489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:36:28.249489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:36:28.249489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:36:28.249489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:36:28.249489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:36:28.249489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:36:28.249489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:36:28.249489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:36:28.249489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:36:28.249489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:36:28.249489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:36:28.250353 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:36:28.250353 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:36:28.250353 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:36:28.250353 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:36:28.250353 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:36:28.250353 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:36:28.250353 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:36:28.250353 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:36:28.250353 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:36:28.250353 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:36:28.250353 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:36:28.250353 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:36:28.256779 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 17/30] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:36:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:36:28.263970 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d25714-f658-4537-b7da-e6b035c204b3 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:28.266710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-62ca2d67-d19b-42c1-8e59-75093a12bb6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:36:28.266710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:36:28.266710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:36:28.266710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:36:28.266710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:36:28.266710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:36:28.266710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:36:28.266710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:36:28.266710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:36:28.266710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:36:28.266710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:36:28.266710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:36:28.266710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:36:28.266710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:36:28.266710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:36:28.266710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:36:28.267566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:36:28.267566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:36:28.267566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:36:28.267566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:36:28.267566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:36:28.267566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:36:28.267566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:36:28.267566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:36:28.267566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:36:28.267566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:36:28.267566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:36:28.267566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:36:28.267566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:36:28.267566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:36:28.267566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:36:28.267566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:36:28.267566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:36:28.269249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:36:28.269249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:36:28.269249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:36:28.269249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 14/31] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:36:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:36:28.276019 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca2d67-d19b-42c1-8e59-75093a12bb6d None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:28.290137 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:28.381522 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:28.399048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d25714-f658-4537-b7da-e6b035c204b3 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:28.434822 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322af81-2725-49c6-9eba-dd69daa5b076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:28.435808 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322af81-2725-49c6-9eba-dd69daa5b076 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:28.436264 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322af81-2725-49c6-9eba-dd69daa5b076 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:28.436705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322af81-2725-49c6-9eba-dd69daa5b076 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:28.437198 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca2d67-d19b-42c1-8e59-75093a12bb6d None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:28.437785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7322af81-2725-49c6-9eba-dd69daa5b076 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:28.438272 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7322af81-2725-49c6-9eba-dd69daa5b076 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:28.439881 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7322af81-2725-49c6-9eba-dd69daa5b076 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:28.449123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 18/32] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:36:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:28.450259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322af81-2725-49c6-9eba-dd69daa5b076 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:28.473013 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321c203-1fce-4162-a7a8-464c7092b594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:28.473565 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321c203-1fce-4162-a7a8-464c7092b594 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:28.473921 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321c203-1fce-4162-a7a8-464c7092b594 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:28.474259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321c203-1fce-4162-a7a8-464c7092b594 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:28.475015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c321c203-1fce-4162-a7a8-464c7092b594 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:28.475410 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c321c203-1fce-4162-a7a8-464c7092b594 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:28.476936 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c321c203-1fce-4162-a7a8-464c7092b594 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:28.496858 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 15/33] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:28.496858 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321c203-1fce-4162-a7a8-464c7092b594 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:28.562660 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322af81-2725-49c6-9eba-dd69daa5b076 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:28.564713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:28.600602 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a28597-439d-47d5-b02a-69146a7b0f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:28.601278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321c203-1fce-4162-a7a8-464c7092b594 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:28.601741 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a28597-439d-47d5-b02a-69146a7b0f54 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:28.602194 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a28597-439d-47d5-b02a-69146a7b0f54 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:28.602641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a28597-439d-47d5-b02a-69146a7b0f54 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:28.603558 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a28597-439d-47d5-b02a-69146a7b0f54 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:28.605198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a28597-439d-47d5-b02a-69146a7b0f54 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:28.640282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc25cb6-eb99-4a84-ac5f-66f5cf036014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:28.641129 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc25cb6-eb99-4a84-ac5f-66f5cf036014 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:28.641448 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc25cb6-eb99-4a84-ac5f-66f5cf036014 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:28.641788 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc25cb6-eb99-4a84-ac5f-66f5cf036014 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:28.642542 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc25cb6-eb99-4a84-ac5f-66f5cf036014 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:28.644041 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc25cb6-eb99-4a84-ac5f-66f5cf036014 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:28.742596 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 19/34] 10.222.0.71 () {60 vars in 1131 bytes} [Fri Aug 26 08:36:28 2022] POST /identity/v3/roles => generated 225 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:36:28.744502 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a28597-439d-47d5-b02a-69146a7b0f54 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:28.768080 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:28.786022 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 16/35] 10.222.0.71 () {60 vars in 1131 bytes} [Fri Aug 26 08:36:28 2022] POST /identity/v3/roles => generated 219 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:36:28.787217 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc25cb6-eb99-4a84-ac5f-66f5cf036014 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:28.890015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a28597-439d-47d5-b02a-69146a7b0f54 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:28.916502 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc25cb6-eb99-4a84-ac5f-66f5cf036014 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:28.934016 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7596d708-b716-44d7-a471-d67e030cc85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:28.935188 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596d708-b716-44d7-a471-d67e030cc85a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:28.935687 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596d708-b716-44d7-a471-d67e030cc85a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:28.936162 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596d708-b716-44d7-a471-d67e030cc85a None admin] PATH_INFO: `/v3/users/668b82af2c1345cca572ef92fff86ff2` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:28.938064 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7596d708-b716-44d7-a471-d67e030cc85a None admin] RBAC: Authorizing `identity:get_user(user_id=668b82af2c1345cca572ef92fff86ff2)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:28.940653 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7596d708-b716-44d7-a471-d67e030cc85a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:28.943144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 20/36] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:36:28 2022] GET /identity/v3/users/668b82af2c1345cca572ef92fff86ff2 => generated 247 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:28.944979 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cec8cd-ecf8-468e-8e47-bf063e83165d None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:28.945492 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cec8cd-ecf8-468e-8e47-bf063e83165d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:28.945959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cec8cd-ecf8-468e-8e47-bf063e83165d None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:28.947734 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 21/37] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:28.952590 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cec8cd-ecf8-468e-8e47-bf063e83165d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:28.959352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:28.964508 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967c276-0126-4abd-afa5-7e7beefba044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:28.965160 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967c276-0126-4abd-afa5-7e7beefba044 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:28.965443 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967c276-0126-4abd-afa5-7e7beefba044 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:28.966100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967c276-0126-4abd-afa5-7e7beefba044 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:28.966787 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4967c276-0126-4abd-afa5-7e7beefba044 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:28.968627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4967c276-0126-4abd-afa5-7e7beefba044 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:29.017392 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 17/38] 10.222.0.71 () {60 vars in 1141 bytes} [Fri Aug 26 08:36:28 2022] POST /identity/v3/projects => generated 301 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:36:29.021279 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f71f6-37ff-4a57-a4ca-d4e238abb52b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:29.021565 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f71f6-37ff-4a57-a4ca-d4e238abb52b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:29.021807 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f71f6-37ff-4a57-a4ca-d4e238abb52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:29.044003 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-1d4f71f6-37ff-4a57-a4ca-d4e238abb52b None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:29.045820 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1d4f71f6-37ff-4a57-a4ca-d4e238abb52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:29.060984 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:29.096997 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cec8cd-ecf8-468e-8e47-bf063e83165d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:29.149275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9f5dd-d12a-4c53-bf7d-010b6e607119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:29.150148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9f5dd-d12a-4c53-bf7d-010b6e607119 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:29.150312 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9f5dd-d12a-4c53-bf7d-010b6e607119 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:29.150468 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9f5dd-d12a-4c53-bf7d-010b6e607119 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:29.151756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea9f5dd-d12a-4c53-bf7d-010b6e607119 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:29.153366 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea9f5dd-d12a-4c53-bf7d-010b6e607119 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:29.156501 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 22/39] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:36:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:29.157728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa238a-c558-4cc0-b0f6-458dd8fbd823 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:29.158010 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa238a-c558-4cc0-b0f6-458dd8fbd823 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:29.158283 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa238a-c558-4cc0-b0f6-458dd8fbd823 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:29.161170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 18/40] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:36:29.164388 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 23/41] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:29.168020 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f71f6-37ff-4a57-a4ca-d4e238abb52b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:29.172193 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d7cc7-4b23-41f3-a4f1-09a2a2f882fd None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:29.172862 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d7cc7-4b23-41f3-a4f1-09a2a2f882fd None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:29.173359 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d7cc7-4b23-41f3-a4f1-09a2a2f882fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:29.194148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-763d7cc7-4b23-41f3-a4f1-09a2a2f882fd None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:29.195791 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-763d7cc7-4b23-41f3-a4f1-09a2a2f882fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:29.312805 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 24/42] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 08:36:29.357390 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f71f6-37ff-4a57-a4ca-d4e238abb52b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:29.379616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:29.429368 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657f13c-d327-4d94-84f1-fbe98fc68041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:29.430343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657f13c-d327-4d94-84f1-fbe98fc68041 None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:29.430652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657f13c-d327-4d94-84f1-fbe98fc68041 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:29.430973 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657f13c-d327-4d94-84f1-fbe98fc68041 None admin] PATH_INFO: `/v3/domains/default/users/668b82af2c1345cca572ef92fff86ff2/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:29.433640 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9657f13c-d327-4d94-84f1-fbe98fc68041 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=668b82af2c1345cca572ef92fff86ff2, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:29.435849 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9657f13c-d327-4d94-84f1-fbe98fc68041 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:29.457561 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 19/43] 10.222.0.71 () {58 vars in 1361 bytes} [Fri Aug 26 08:36:29 2022] PUT /identity/v3/domains/default/users/668b82af2c1345cca572ef92fff86ff2/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:36:29.527938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d7cc7-4b23-41f3-a4f1-09a2a2f882fd None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:29.617125 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657f13c-d327-4d94-84f1-fbe98fc68041 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:29.642739 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:29.676329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d7cc7-4b23-41f3-a4f1-09a2a2f882fd None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:29.715988 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d8f47-3bd0-4704-90d2-b34fa7e36a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:29.716680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d8f47-3bd0-4704-90d2-b34fa7e36a40 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:29.717198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d8f47-3bd0-4704-90d2-b34fa7e36a40 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:29.717517 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d8f47-3bd0-4704-90d2-b34fa7e36a40 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:29.718230 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d8f47-3bd0-4704-90d2-b34fa7e36a40 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:29.719919 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d8f47-3bd0-4704-90d2-b34fa7e36a40 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:29.735908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 25/44] 10.222.0.71 () {60 vars in 1131 bytes} [Fri Aug 26 08:36:29 2022] POST /identity/v3/roles => generated 223 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:36:29.876200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657f13c-d327-4d94-84f1-fbe98fc68041 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:29.922564 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa073f7-5e89-4ce6-aa36-c224747b79c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:29.923296 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa073f7-5e89-4ce6-aa36-c224747b79c7 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:29.923575 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa073f7-5e89-4ce6-aa36-c224747b79c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:29.923868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa073f7-5e89-4ce6-aa36-c224747b79c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:29.925057 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa073f7-5e89-4ce6-aa36-c224747b79c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:29.926541 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa073f7-5e89-4ce6-aa36-c224747b79c7 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:29.928615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 20/45] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:36:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:36:29.934690 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d8f47-3bd0-4704-90d2-b34fa7e36a40 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:30.060836 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d8f47-3bd0-4704-90d2-b34fa7e36a40 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:30.109396 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43b2e2-2d2f-49ba-9d7a-bdb43319101e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:30.110137 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43b2e2-2d2f-49ba-9d7a-bdb43319101e None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:30.110422 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43b2e2-2d2f-49ba-9d7a-bdb43319101e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:30.110698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43b2e2-2d2f-49ba-9d7a-bdb43319101e None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:30.111411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd43b2e2-2d2f-49ba-9d7a-bdb43319101e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:30.113225 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd43b2e2-2d2f-49ba-9d7a-bdb43319101e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:30.119000 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:30.157325 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 26/46] 10.222.0.71 () {60 vars in 1141 bytes} [Fri Aug 26 08:36:29 2022] POST /identity/v3/projects => generated 312 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:36:31.438710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e869c-a8b0-4760-b2fd-aab408a8691b None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:31.439327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e869c-a8b0-4760-b2fd-aab408a8691b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:31.439327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e869c-a8b0-4760-b2fd-aab408a8691b None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:31.440725 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 21/47] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:31.446355 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b34fce-e1e3-4c54-85d2-e37e8a6f2ca5 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:31.446808 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b34fce-e1e3-4c54-85d2-e37e8a6f2ca5 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:31.446937 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b34fce-e1e3-4c54-85d2-e37e8a6f2ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:31.467325 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e9b34fce-e1e3-4c54-85d2-e37e8a6f2ca5 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:31.468551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e9b34fce-e1e3-4c54-85d2-e37e8a6f2ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:31.542672 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 27/48] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:36:31.804936 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e869c-a8b0-4760-b2fd-aab408a8691b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:31.849723 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:31.927266 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e869c-a8b0-4760-b2fd-aab408a8691b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:31.965729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55100b-c87b-43a9-9e50-8537436d6a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:31.966388 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55100b-c87b-43a9-9e50-8537436d6a14 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:31.966673 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55100b-c87b-43a9-9e50-8537436d6a14 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:31.966984 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55100b-c87b-43a9-9e50-8537436d6a14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:31.978298 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a55100b-c87b-43a9-9e50-8537436d6a14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:36:31.983233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-3a55100b-c87b-43a9-9e50-8537436d6a14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:36:31.983233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:36:31.983233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:36:31.983233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:36:31.983233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:36:31.983233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:36:31.983233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:36:31.983233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:36:31.983233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:36:31.983233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:36:31.983233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:36:31.983233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:36:31.983233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:36:31.983233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:36:31.983233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:36:31.983233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:36:31.984533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:36:31.984533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:36:31.984533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:36:31.984533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:36:31.984533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:36:31.984533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:36:31.984533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:36:31.984533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:36:31.984533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:36:31.984533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:36:31.984533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:36:31.984533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:36:31.984533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:36:31.984533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:36:31.984533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:36:31.984533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:36:31.985352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:36:31.985352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:36:31.985352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:36:31.985352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:36:31.985352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:36:31.985352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:36:31.985352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:36:31.985352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:36:31.985352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:36:31.985352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:36:31.985352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:36:31.985352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:36:31.985352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 22/49] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:36:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:36:31.991239 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b34fce-e1e3-4c54-85d2-e37e8a6f2ca5 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:32.104690 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b34fce-e1e3-4c54-85d2-e37e8a6f2ca5 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:32.110997 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1ea40-1229-4d53-8ee8-51441b3ab56e None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:32.111243 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1ea40-1229-4d53-8ee8-51441b3ab56e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:32.111566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1ea40-1229-4d53-8ee8-51441b3ab56e None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:32.113315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 23/50] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:32.119384 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d5afa-3692-4e0f-a778-a30a1a4d774f None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:32.119666 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d5afa-3692-4e0f-a778-a30a1a4d774f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:32.119937 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d5afa-3692-4e0f-a778-a30a1a4d774f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:32.123572 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:32.139058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-4b0d5afa-3692-4e0f-a778-a30a1a4d774f None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:32.140280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4b0d5afa-3692-4e0f-a778-a30a1a4d774f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:32.144254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb12978-2801-4bd0-be33-f7e5bac316a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:32.144911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb12978-2801-4bd0-be33-f7e5bac316a9 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:32.145332 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb12978-2801-4bd0-be33-f7e5bac316a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:32.145456 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb12978-2801-4bd0-be33-f7e5bac316a9 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:32.146135 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb12978-2801-4bd0-be33-f7e5bac316a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:32.146438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb12978-2801-4bd0-be33-f7e5bac316a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:32.147899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb12978-2801-4bd0-be33-f7e5bac316a9 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:32.157581 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 28/51] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:36:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:32.163327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb12978-2801-4bd0-be33-f7e5bac316a9 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:32.219782 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 24/52] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:36:32.274365 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb12978-2801-4bd0-be33-f7e5bac316a9 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:32.309147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e6188-8886-4b49-8198-3c5e937673a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:32.309797 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e6188-8886-4b49-8198-3c5e937673a9 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:32.310074 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e6188-8886-4b49-8198-3c5e937673a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:32.310348 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e6188-8886-4b49-8198-3c5e937673a9 None admin] PATH_INFO: `/v3/users/668b82af2c1345cca572ef92fff86ff2` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:32.311530 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e6188-8886-4b49-8198-3c5e937673a9 None admin] RBAC: Authorizing `identity:get_user(user_id=668b82af2c1345cca572ef92fff86ff2)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:32.313033 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e6188-8886-4b49-8198-3c5e937673a9 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:32.314635 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 29/53] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:36:32 2022] GET /identity/v3/users/668b82af2c1345cca572ef92fff86ff2 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:36:32.320330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d5afa-3692-4e0f-a778-a30a1a4d774f None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:32.348446 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:32.452782 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d5afa-3692-4e0f-a778-a30a1a4d774f None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:32.491371 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2100964-6bb0-4968-8b96-5a891fc19673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:32.492356 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2100964-6bb0-4968-8b96-5a891fc19673 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:32.492790 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2100964-6bb0-4968-8b96-5a891fc19673 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:32.493218 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2100964-6bb0-4968-8b96-5a891fc19673 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:32.494452 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2100964-6bb0-4968-8b96-5a891fc19673 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:32.495802 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e6188-8886-4b49-8198-3c5e937673a9 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:32.496280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2100964-6bb0-4968-8b96-5a891fc19673 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:32.498163 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 25/54] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:36:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:32.504185 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2100964-6bb0-4968-8b96-5a891fc19673 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:32.544773 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:32.603135 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e6188-8886-4b49-8198-3c5e937673a9 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:32.608031 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:32.617350 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2100964-6bb0-4968-8b96-5a891fc19673 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:32.638168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78565483-0d16-4dc3-8eaf-1eae8952f280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:32.638808 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78565483-0d16-4dc3-8eaf-1eae8952f280 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:32.639103 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78565483-0d16-4dc3-8eaf-1eae8952f280 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:32.639390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78565483-0d16-4dc3-8eaf-1eae8952f280 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:32.640510 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78565483-0d16-4dc3-8eaf-1eae8952f280 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:32.642020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78565483-0d16-4dc3-8eaf-1eae8952f280 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:32.643764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 30/55] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:36:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:36:32.649467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78565483-0d16-4dc3-8eaf-1eae8952f280 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:32.653370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8fb0c-9791-409c-be50-7b52d1c292e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:32.654004 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8fb0c-9791-409c-be50-7b52d1c292e7 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:32.654307 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8fb0c-9791-409c-be50-7b52d1c292e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:32.654566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8fb0c-9791-409c-be50-7b52d1c292e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:32.655360 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df8fb0c-9791-409c-be50-7b52d1c292e7 None admin] RBAC: Adding query filter params (role.id=18c3c8a821b647f7982a955880bd0a9a, scope.domain.id=default, user.id=668b82af2c1345cca572ef92fff86ff2) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:32.655673 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df8fb0c-9791-409c-be50-7b52d1c292e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:32.657191 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df8fb0c-9791-409c-be50-7b52d1c292e7 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:32.664793 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 26/56] 10.222.0.71 () {56 vars in 1322 bytes} [Fri Aug 26 08:36:32 2022] GET /identity/v3/role_assignments?user.id=668b82af2c1345cca572ef92fff86ff2&scope.domain.id=default&role.id=18c3c8a821b647f7982a955880bd0a9a => generated 530 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:36:32.762170 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78565483-0d16-4dc3-8eaf-1eae8952f280 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:32.778191 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:32.803647 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e6851-369f-4ce0-ade5-1da0f507b32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:32.804299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e6851-369f-4ce0-ade5-1da0f507b32b None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:32.804578 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e6851-369f-4ce0-ade5-1da0f507b32b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:32.804850 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e6851-369f-4ce0-ade5-1da0f507b32b None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:32.805511 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25e6851-369f-4ce0-ade5-1da0f507b32b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:32.807031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25e6851-369f-4ce0-ade5-1da0f507b32b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:32.837597 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 31/57] 10.222.0.71 () {60 vars in 1140 bytes} [Fri Aug 26 08:36:32 2022] POST /identity/v3/projects => generated 298 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:36:34.797456 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649760b1-82eb-46fa-97d4-6cbc8f8491a8 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:34.798004 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649760b1-82eb-46fa-97d4-6cbc8f8491a8 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:34.798004 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649760b1-82eb-46fa-97d4-6cbc8f8491a8 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:34.799387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 27/58] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:34.805020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea72c80-bbbd-4afa-af3b-0d07eb686a5c None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:34.805494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea72c80-bbbd-4afa-af3b-0d07eb686a5c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:34.805578 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea72c80-bbbd-4afa-af3b-0d07eb686a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:34.932847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-bea72c80-bbbd-4afa-af3b-0d07eb686a5c None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:34.934092 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-bea72c80-bbbd-4afa-af3b-0d07eb686a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:35.008805 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 32/59] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:36:35.278241 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649760b1-82eb-46fa-97d4-6cbc8f8491a8 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:35.393796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649760b1-82eb-46fa-97d4-6cbc8f8491a8 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:35.429995 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6182197-3850-4db7-aa3a-aa6425aaa2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:35.430637 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6182197-3850-4db7-aa3a-aa6425aaa2f6 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:35.430934 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6182197-3850-4db7-aa3a-aa6425aaa2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:35.431214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6182197-3850-4db7-aa3a-aa6425aaa2f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:35.432359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6182197-3850-4db7-aa3a-aa6425aaa2f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:35.433813 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6182197-3850-4db7-aa3a-aa6425aaa2f6 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:35.435442 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 28/60] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:36:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:35.442942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea72c80-bbbd-4afa-af3b-0d07eb686a5c None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:35.548325 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:35.557669 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea72c80-bbbd-4afa-af3b-0d07eb686a5c None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:35.604825 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0169131f-8b85-4e13-83fd-00e7f6880e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:35.605460 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169131f-8b85-4e13-83fd-00e7f6880e8d None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:35.605745 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169131f-8b85-4e13-83fd-00e7f6880e8d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:35.606014 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169131f-8b85-4e13-83fd-00e7f6880e8d None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:35.606672 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0169131f-8b85-4e13-83fd-00e7f6880e8d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:35.608207 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0169131f-8b85-4e13-83fd-00e7f6880e8d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:35.637594 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 33/61] 10.222.0.71 () {60 vars in 1132 bytes} [Fri Aug 26 08:36:35 2022] POST /identity/v3/users => generated 275 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:36:39.158082 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b393275-cde4-4506-b714-3e717a3a7c11 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:39.158739 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b393275-cde4-4506-b714-3e717a3a7c11 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:39.158739 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b393275-cde4-4506-b714-3e717a3a7c11 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:39.160338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 29/62] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:39.165436 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46f532-7b68-4ca3-a7cb-1f28d8dd09b9 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:39.166304 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46f532-7b68-4ca3-a7cb-1f28d8dd09b9 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:39.166782 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46f532-7b68-4ca3-a7cb-1f28d8dd09b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:39.179256 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d518b-4404-4b5d-a42e-ab0073d84638 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:39.179809 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d518b-4404-4b5d-a42e-ab0073d84638 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:39.180309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d518b-4404-4b5d-a42e-ab0073d84638 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:39.182223 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 30/63] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:39.188158 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ac829-4450-4f23-9f55-467f9e4c651a None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:39.188753 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ac829-4450-4f23-9f55-467f9e4c651a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:39.189239 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ac829-4450-4f23-9f55-467f9e4c651a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:39.196227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-eb46f532-7b68-4ca3-a7cb-1f28d8dd09b9 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:39.198249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-eb46f532-7b68-4ca3-a7cb-1f28d8dd09b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:39.210313 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-cf3ac829-4450-4f23-9f55-467f9e4c651a None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:39.212233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-cf3ac829-4450-4f23-9f55-467f9e4c651a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:39.236178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:39.293208 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 31/64] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 08:36:39.299010 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e730e0-78c4-4964-8d8d-65d70353d8a4 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:39.299010 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e730e0-78c4-4964-8d8d-65d70353d8a4 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:39.299010 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e730e0-78c4-4964-8d8d-65d70353d8a4 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:39.299010 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 32/65] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:39.304464 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 34/66] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:36:39.306411 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d673c-8ea0-41bf-bb5d-cd004ebc2dec None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:39.306700 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d673c-8ea0-41bf-bb5d-cd004ebc2dec None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:39.307010 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d673c-8ea0-41bf-bb5d-cd004ebc2dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:39.339834 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c79d673c-8ea0-41bf-bb5d-cd004ebc2dec None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:39.341338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c79d673c-8ea0-41bf-bb5d-cd004ebc2dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:39.434405 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 33/67] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:36:39.597192 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0567cd7-671a-409c-8a99-91629b8a5bc3 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:39.597630 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0567cd7-671a-409c-8a99-91629b8a5bc3 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:39.597844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0567cd7-671a-409c-8a99-91629b8a5bc3 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:39.599206 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 35/68] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:39.600507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d673c-8ea0-41bf-bb5d-cd004ebc2dec None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:39.622508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e693f-d0f5-45bf-8775-3cb6c706a52e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:39.623227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e693f-d0f5-45bf-8775-3cb6c706a52e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:39.623824 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e693f-d0f5-45bf-8775-3cb6c706a52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:39.650172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-0f4e693f-d0f5-45bf-8775-3cb6c706a52e None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:39.653004 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-0f4e693f-d0f5-45bf-8775-3cb6c706a52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:39.730406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:39.742263 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 36/69] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 08:36:39.744010 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3af47-ce8d-41f7-af97-53f212dab785 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:39.744237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3af47-ce8d-41f7-af97-53f212dab785 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:39.744516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3af47-ce8d-41f7-af97-53f212dab785 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:39.745799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 37/70] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:39.746677 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3af47-ce8d-41f7-af97-53f212dab785 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:39.758574 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d673c-8ea0-41bf-bb5d-cd004ebc2dec None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:39.766204 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:39.801130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc217c9e-26be-42b5-b662-122b1a662744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:39.801775 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc217c9e-26be-42b5-b662-122b1a662744 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:39.802060 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc217c9e-26be-42b5-b662-122b1a662744 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:39.802338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc217c9e-26be-42b5-b662-122b1a662744 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:39.813391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc217c9e-26be-42b5-b662-122b1a662744 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:36:39.818206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-bc217c9e-26be-42b5-b662-122b1a662744 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:36:39.818206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:36:39.818206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:36:39.818206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:36:39.818206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:36:39.818206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:36:39.818206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:36:39.818206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:36:39.818206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:36:39.818206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:36:39.818206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:36:39.818206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:36:39.818206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:36:39.818206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:36:39.818206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:36:39.818206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:36:39.819268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:36:39.819268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:36:39.819268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:36:39.819268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:36:39.819268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:36:39.819268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:36:39.819268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:36:39.819268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:36:39.819268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:36:39.819268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:36:39.819268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:36:39.819268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:36:39.819268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:36:39.819268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:36:39.819268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:36:39.819268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:36:39.820203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:36:39.820203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:36:39.820203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:36:39.820203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:36:39.820203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:36:39.820203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:36:39.820203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:36:39.820203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:36:39.820203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:36:39.820203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:36:39.820203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:36:39.820203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:36:39.820203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 34/71] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:36:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:36:39.821002 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890cb573-20f2-422f-b887-7bc6070adb13 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:39.821426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890cb573-20f2-422f-b887-7bc6070adb13 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:39.821557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890cb573-20f2-422f-b887-7bc6070adb13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:39.839792 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-890cb573-20f2-422f-b887-7bc6070adb13 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:39.841129 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-890cb573-20f2-422f-b887-7bc6070adb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:39.879880 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3af47-ce8d-41f7-af97-53f212dab785 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:39.927833 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 35/72] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:36:39.928384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5673a50-73c7-4910-aca9-88bac5975ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:39.929195 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5673a50-73c7-4910-aca9-88bac5975ed9 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:39.929480 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5673a50-73c7-4910-aca9-88bac5975ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:39.929748 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5673a50-73c7-4910-aca9-88bac5975ed9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:39.930089 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890cb573-20f2-422f-b887-7bc6070adb13 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:39.942829 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5673a50-73c7-4910-aca9-88bac5975ed9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:36:39.947935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-f5673a50-73c7-4910-aca9-88bac5975ed9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:36:39.947935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:36:39.947935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:36:39.947935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:36:39.947935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:36:39.947935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:36:39.947935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:36:39.947935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:36:39.947935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:36:39.947935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:36:39.947935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:36:39.947935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:36:39.947935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:36:39.947935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:36:39.947935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:36:39.947935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:36:39.948810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:36:39.948810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:36:39.948810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:36:39.948810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:36:39.948810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:36:39.948810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:36:39.948810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:36:39.948810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:36:39.948810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:36:39.948810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:36:39.948810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:36:39.948810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:36:39.948810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:36:39.948810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:36:39.948810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:36:39.948810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:36:39.949583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:36:39.949583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:36:39.949583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:36:39.949583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:36:39.949583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:36:39.949583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:36:39.949583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:36:39.949583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:36:39.949583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:36:39.949583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:36:39.949583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:36:39.949583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:36:39.949583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 38/73] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:36:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:36:39.950227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5673a50-73c7-4910-aca9-88bac5975ed9 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:40.033845 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:40.036812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:40.068088 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890cb573-20f2-422f-b887-7bc6070adb13 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:40.068386 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5673a50-73c7-4910-aca9-88bac5975ed9 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:40.108545 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1246ede-7cf1-4869-8cfb-06ba70ac4362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:40.109295 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1246ede-7cf1-4869-8cfb-06ba70ac4362 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:40.109630 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1246ede-7cf1-4869-8cfb-06ba70ac4362 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:40.109996 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1246ede-7cf1-4869-8cfb-06ba70ac4362 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:40.110678 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1246ede-7cf1-4869-8cfb-06ba70ac4362 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:40.111010 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1246ede-7cf1-4869-8cfb-06ba70ac4362 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:40.111163 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441f9f9-b84e-4bdf-abc7-802ede321af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:40.111803 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441f9f9-b84e-4bdf-abc7-802ede321af4 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:40.112091 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441f9f9-b84e-4bdf-abc7-802ede321af4 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:40.112369 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441f9f9-b84e-4bdf-abc7-802ede321af4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:40.112453 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1246ede-7cf1-4869-8cfb-06ba70ac4362 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:40.126567 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 39/74] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:36:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:40.126567 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1246ede-7cf1-4869-8cfb-06ba70ac4362 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:40.129095 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441f9f9-b84e-4bdf-abc7-802ede321af4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:36:40.133546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-f441f9f9-b84e-4bdf-abc7-802ede321af4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 08:36:40.133546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:36:40.133546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:36:40.133546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:36:40.133546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:36:40.133546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:36:40.133546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:36:40.133546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:36:40.133546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:36:40.133546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:36:40.133546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:36:40.133546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:36:40.133546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:36:40.133546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:36:40.133546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:36:40.133546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:36:40.134408 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:36:40.134408 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:36:40.134408 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:36:40.134408 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:36:40.134408 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:36:40.134408 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:36:40.134408 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:36:40.134408 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:36:40.134408 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:36:40.134408 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:36:40.134408 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:36:40.134408 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:36:40.134408 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:36:40.134408 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:36:40.134408 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:36:40.134408 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:36:40.135169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:36:40.135169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:36:40.135169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:36:40.135169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:36:40.135169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:36:40.135169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:36:40.135169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:36:40.135169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:36:40.135169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:36:40.135169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:36:40.135169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 08:36:40.135169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:36:40.135169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 36/75] 10.222.0.71 () {56 vars in 1114 bytes} [Fri Aug 26 08:36:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:36:40.135802 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441f9f9-b84e-4bdf-abc7-802ede321af4 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:40.241911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1246ede-7cf1-4869-8cfb-06ba70ac4362 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:40.246547 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441f9f9-b84e-4bdf-abc7-802ede321af4 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:40.277322 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f19ac-a93d-4fff-a156-22698aa12345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:40.277938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f19ac-a93d-4fff-a156-22698aa12345 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:40.278219 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f19ac-a93d-4fff-a156-22698aa12345 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:40.278490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f19ac-a93d-4fff-a156-22698aa12345 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:40.279195 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f19ac-a93d-4fff-a156-22698aa12345 None admin] RBAC: Adding query filter params (name=member) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:40.279534 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f19ac-a93d-4fff-a156-22698aa12345 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:40.280900 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f19ac-a93d-4fff-a156-22698aa12345 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:40.281005 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7c7fd-ae96-41a8-87e9-c66c7c586874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:40.281875 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7c7fd-ae96-41a8-87e9-c66c7c586874 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:40.282007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7c7fd-ae96-41a8-87e9-c66c7c586874 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:40.282293 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7c7fd-ae96-41a8-87e9-c66c7c586874 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:40.282400 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:40.285557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:40.290741 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 40/76] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:36:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:40.291636 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f19ac-a93d-4fff-a156-22698aa12345 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:40.295092 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f7c7fd-ae96-41a8-87e9-c66c7c586874 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:36:40.300449 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-17f7c7fd-ae96-41a8-87e9-c66c7c586874 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:36:40.300449 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:36:40.300449 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:36:40.300449 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:36:40.300449 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:36:40.300449 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:36:40.300449 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:36:40.300449 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:36:40.300449 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:36:40.300449 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:36:40.300449 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:36:40.300449 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:36:40.300449 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:36:40.300449 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:36:40.300449 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:36:40.300449 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:36:40.301514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:36:40.301514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:36:40.301514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:36:40.301514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:36:40.301514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:36:40.301514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:36:40.301514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:36:40.301514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:36:40.301514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:36:40.301514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:36:40.301514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:36:40.301514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:36:40.301514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:36:40.301514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:36:40.301514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:36:40.301514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:36:40.302529 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:36:40.302529 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:36:40.302529 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:36:40.302529 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:36:40.302529 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:36:40.302529 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:36:40.302529 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:36:40.302529 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:36:40.302529 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:36:40.302529 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:36:40.302529 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:36:40.302529 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:36:40.302529 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 37/77] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:36:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:36:40.302529 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7c7fd-ae96-41a8-87e9-c66c7c586874 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:40.413708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f19ac-a93d-4fff-a156-22698aa12345 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:40.417489 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7c7fd-ae96-41a8-87e9-c66c7c586874 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:40.463363 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a7b1d-0f4b-4a43-bc69-c0e2f8bc8a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:40.463743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb0961-b2b5-4b50-bdb2-e8b52e7f17b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:40.464356 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a7b1d-0f4b-4a43-bc69-c0e2f8bc8a4c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:40.464707 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb0961-b2b5-4b50-bdb2-e8b52e7f17b7 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:40.464851 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a7b1d-0f4b-4a43-bc69-c0e2f8bc8a4c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:40.465160 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb0961-b2b5-4b50-bdb2-e8b52e7f17b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:40.465423 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a7b1d-0f4b-4a43-bc69-c0e2f8bc8a4c None admin] PATH_INFO: `/v3/users/668b82af2c1345cca572ef92fff86ff2` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:40.465650 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb0961-b2b5-4b50-bdb2-e8b52e7f17b7 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:40.466767 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb0961-b2b5-4b50-bdb2-e8b52e7f17b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:40.467276 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb0961-b2b5-4b50-bdb2-e8b52e7f17b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:40.467379 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a7b1d-0f4b-4a43-bc69-c0e2f8bc8a4c None admin] RBAC: Authorizing `identity:get_user(user_id=668b82af2c1345cca572ef92fff86ff2)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:40.469615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb0961-b2b5-4b50-bdb2-e8b52e7f17b7 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:40.469738 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a7b1d-0f4b-4a43-bc69-c0e2f8bc8a4c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:40.477440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 41/78] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:36:40 2022] GET /identity/v3/users/668b82af2c1345cca572ef92fff86ff2 => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:40.477440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a7b1d-0f4b-4a43-bc69-c0e2f8bc8a4c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:40.490278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 38/79] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:36:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:40.491826 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb0961-b2b5-4b50-bdb2-e8b52e7f17b7 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:40.562491 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:40.562681 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:40.603850 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a7b1d-0f4b-4a43-bc69-c0e2f8bc8a4c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:40.608166 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb0961-b2b5-4b50-bdb2-e8b52e7f17b7 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:40.645023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15022f6b-f9f2-4e6e-b66b-db78bb906c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:40.645693 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15022f6b-f9f2-4e6e-b66b-db78bb906c5b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:40.646820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15022f6b-f9f2-4e6e-b66b-db78bb906c5b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:40.646820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15022f6b-f9f2-4e6e-b66b-db78bb906c5b None admin] PATH_INFO: `/v3/domains/default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:40.647829 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15022f6b-f9f2-4e6e-b66b-db78bb906c5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:40.649446 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15022f6b-f9f2-4e6e-b66b-db78bb906c5b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:40.650387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9638c3ba-4c02-420a-bf25-ce2b166df4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:40.651103 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 42/80] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:36:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:40.651272 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638c3ba-4c02-420a-bf25-ce2b166df4e1 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:40.651566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638c3ba-4c02-420a-bf25-ce2b166df4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:40.651896 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638c3ba-4c02-420a-bf25-ce2b166df4e1 None admin] PATH_INFO: `/v3/users/748c27d9ec1c4f29912ec71cb6f8863c` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:40.651994 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15022f6b-f9f2-4e6e-b66b-db78bb906c5b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:40.669388 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9638c3ba-4c02-420a-bf25-ce2b166df4e1 None admin] RBAC: Authorizing `identity:get_user(user_id=748c27d9ec1c4f29912ec71cb6f8863c)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:40.671548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9638c3ba-4c02-420a-bf25-ce2b166df4e1 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:40.682220 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 39/81] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:36:40 2022] GET /identity/v3/users/748c27d9ec1c4f29912ec71cb6f8863c => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:40.682220 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9638c3ba-4c02-420a-bf25-ce2b166df4e1 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:40.777967 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15022f6b-f9f2-4e6e-b66b-db78bb906c5b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:40.791063 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9638c3ba-4c02-420a-bf25-ce2b166df4e1 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:40.811580 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:40.820804 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:40.827505 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eebaef-ccb3-44fb-b8d6-9cd290564e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:40.828427 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eebaef-ccb3-44fb-b8d6-9cd290564e8c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:40.828731 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eebaef-ccb3-44fb-b8d6-9cd290564e8c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:40.829016 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eebaef-ccb3-44fb-b8d6-9cd290564e8c None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:40.829700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eebaef-ccb3-44fb-b8d6-9cd290564e8c None admin] RBAC: Adding query filter params (name=member) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:40.830002 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eebaef-ccb3-44fb-b8d6-9cd290564e8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:40.831451 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eebaef-ccb3-44fb-b8d6-9cd290564e8c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:40.834467 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e85002-5fdd-463a-8fd2-596b40a4ac67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:40.835127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e85002-5fdd-463a-8fd2-596b40a4ac67 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:40.835412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e85002-5fdd-463a-8fd2-596b40a4ac67 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:40.835691 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e85002-5fdd-463a-8fd2-596b40a4ac67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:40.842938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 43/82] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:36:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:40.843804 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eebaef-ccb3-44fb-b8d6-9cd290564e8c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:40.914355 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e85002-5fdd-463a-8fd2-596b40a4ac67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:40.916002 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e85002-5fdd-463a-8fd2-596b40a4ac67 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:40.954293 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eebaef-ccb3-44fb-b8d6-9cd290564e8c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:41.005824 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ad29e-f491-4899-abf4-2d5de2162cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:41.006505 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ad29e-f491-4899-abf4-2d5de2162cac None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:41.006807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ad29e-f491-4899-abf4-2d5de2162cac None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:41.007138 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ad29e-f491-4899-abf4-2d5de2162cac None admin] PATH_INFO: `/v3/users/748c27d9ec1c4f29912ec71cb6f8863c` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:41.008567 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ad29e-f491-4899-abf4-2d5de2162cac None admin] RBAC: Authorizing `identity:get_user(user_id=748c27d9ec1c4f29912ec71cb6f8863c)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:41.010330 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ad29e-f491-4899-abf4-2d5de2162cac None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:41.012154 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 44/83] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:36:40 2022] GET /identity/v3/users/748c27d9ec1c4f29912ec71cb6f8863c => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:41.012972 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ad29e-f491-4899-abf4-2d5de2162cac None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:41.043280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 40/84] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:36:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:36:41.043280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e85002-5fdd-463a-8fd2-596b40a4ac67 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:41.061987 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:41.082842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:41.132279 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ad29e-f491-4899-abf4-2d5de2162cac None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:41.152660 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e85002-5fdd-463a-8fd2-596b40a4ac67 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:41.170119 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8f6d2-2e7d-4e3f-b300-65d4ca5f5aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:41.170755 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8f6d2-2e7d-4e3f-b300-65d4ca5f5aee None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:41.171070 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8f6d2-2e7d-4e3f-b300-65d4ca5f5aee None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:41.171338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8f6d2-2e7d-4e3f-b300-65d4ca5f5aee None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:41.172023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c8f6d2-2e7d-4e3f-b300-65d4ca5f5aee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:41.173577 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c8f6d2-2e7d-4e3f-b300-65d4ca5f5aee None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:41.195517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db7cc5-d18e-45ba-9ccb-b659a382c39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:41.196296 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db7cc5-d18e-45ba-9ccb-b659a382c39b None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:41.196654 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db7cc5-d18e-45ba-9ccb-b659a382c39b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:41.196856 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db7cc5-d18e-45ba-9ccb-b659a382c39b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:41.197501 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 45/85] 10.222.0.71 () {60 vars in 1132 bytes} [Fri Aug 26 08:36:41 2022] POST /identity/v3/users => generated 289 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:36:41.197501 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8f6d2-2e7d-4e3f-b300-65d4ca5f5aee None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:41.325378 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db7cc5-d18e-45ba-9ccb-b659a382c39b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:41.329244 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db7cc5-d18e-45ba-9ccb-b659a382c39b None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:41.368071 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:41.387547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8f6d2-2e7d-4e3f-b300-65d4ca5f5aee None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:41.411786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:41.438605 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e176b4-6923-4412-9fbb-99719082df1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:41.439715 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e176b4-6923-4412-9fbb-99719082df1a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:41.440186 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e176b4-6923-4412-9fbb-99719082df1a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:41.441755 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e176b4-6923-4412-9fbb-99719082df1a None admin] PATH_INFO: `/v3/users/748c27d9ec1c4f29912ec71cb6f8863c` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:41.442284 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e176b4-6923-4412-9fbb-99719082df1a None admin] RBAC: Authorizing `identity:get_user(user_id=748c27d9ec1c4f29912ec71cb6f8863c)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:41.444802 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e176b4-6923-4412-9fbb-99719082df1a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:41.447119 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 46/86] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:36:41 2022] GET /identity/v3/users/748c27d9ec1c4f29912ec71cb6f8863c => generated 275 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:41.450472 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e176b4-6923-4412-9fbb-99719082df1a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:41.517946 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 41/87] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:36:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:36:41.520432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db7cc5-d18e-45ba-9ccb-b659a382c39b None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:41.603522 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e176b4-6923-4412-9fbb-99719082df1a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:41.672649 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db7cc5-d18e-45ba-9ccb-b659a382c39b None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:41.672882 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c1be4-9182-4547-ada9-7d3a73e0d5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:41.678919 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c1be4-9182-4547-ada9-7d3a73e0d5ba None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:41.678919 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c1be4-9182-4547-ada9-7d3a73e0d5ba None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:41.678919 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c1be4-9182-4547-ada9-7d3a73e0d5ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:41.691501 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:41.749596 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:41.751497 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca39d39-f9fd-492a-aeaf-eacc6e926ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:41.752429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca39d39-f9fd-492a-aeaf-eacc6e926ef0 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:41.752532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca39d39-f9fd-492a-aeaf-eacc6e926ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:41.752758 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca39d39-f9fd-492a-aeaf-eacc6e926ef0 None admin] PATH_INFO: `/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:41.764140 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca39d39-f9fd-492a-aeaf-eacc6e926ef0 None admin] RBAC: Authorizing `identity:get_project(project_id=e55f1cc9f9c9481698d6962c7af7fbfe)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:41.765834 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca39d39-f9fd-492a-aeaf-eacc6e926ef0 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:41.767975 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 42/88] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:36:41 2022] GET /identity/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:41.768214 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c1be4-9182-4547-ada9-7d3a73e0d5ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:41.773498 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca39d39-f9fd-492a-aeaf-eacc6e926ef0 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:41.775608 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c1be4-9182-4547-ada9-7d3a73e0d5ba None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:41.911521 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 47/89] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:36:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:36:41.913862 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c1be4-9182-4547-ada9-7d3a73e0d5ba None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:42.051978 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:42.093548 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c1be4-9182-4547-ada9-7d3a73e0d5ba None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:42.093884 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca39d39-f9fd-492a-aeaf-eacc6e926ef0 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:42.143252 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf71897-b1b4-4475-a0e2-50a78af05cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:42.143894 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf71897-b1b4-4475-a0e2-50a78af05cf0 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:42.144182 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf71897-b1b4-4475-a0e2-50a78af05cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:42.144461 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf71897-b1b4-4475-a0e2-50a78af05cf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:42.224026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c89d42-9193-4f89-a220-572059cdf605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:42.225029 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c89d42-9193-4f89-a220-572059cdf605 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:42.225467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c89d42-9193-4f89-a220-572059cdf605 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:42.225892 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c89d42-9193-4f89-a220-572059cdf605 None admin] PATH_INFO: `/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:42.227902 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c89d42-9193-4f89-a220-572059cdf605 None admin] RBAC: Authorizing `identity:get_project(project_id=e55f1cc9f9c9481698d6962c7af7fbfe)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:42.230256 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c89d42-9193-4f89-a220-572059cdf605 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:42.232973 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 48/90] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:36:41 2022] GET /identity/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe => generated 298 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:42.238614 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c89d42-9193-4f89-a220-572059cdf605 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:42.279856 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:42.306922 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf71897-b1b4-4475-a0e2-50a78af05cf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:42.309575 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf71897-b1b4-4475-a0e2-50a78af05cf0 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:42.434438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c89d42-9193-4f89-a220-572059cdf605 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:42.461108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 43/91] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:36:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 692 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:36:42.462930 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf71897-b1b4-4475-a0e2-50a78af05cf0 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:42.499753 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7facb568-4423-4335-96ae-cd7d517f86f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:42.500854 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7facb568-4423-4335-96ae-cd7d517f86f6 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:42.501244 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7facb568-4423-4335-96ae-cd7d517f86f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:42.501900 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7facb568-4423-4335-96ae-cd7d517f86f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:42.503078 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7facb568-4423-4335-96ae-cd7d517f86f6 None admin] RBAC: Adding query filter params (role.id=18c3c8a821b647f7982a955880bd0a9a, scope.project.id=e55f1cc9f9c9481698d6962c7af7fbfe, user.id=668b82af2c1345cca572ef92fff86ff2) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:42.503733 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7facb568-4423-4335-96ae-cd7d517f86f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:42.506638 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7facb568-4423-4335-96ae-cd7d517f86f6 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:42.522923 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 49/92] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:36:42 2022] GET /identity/v3/role_assignments?user.id=668b82af2c1345cca572ef92fff86ff2&scope.project.id=e55f1cc9f9c9481698d6962c7af7fbfe&role.id=18c3c8a821b647f7982a955880bd0a9a => generated 259 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:42.522923 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7facb568-4423-4335-96ae-cd7d517f86f6 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:42.586752 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:42.609241 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf71897-b1b4-4475-a0e2-50a78af05cf0 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:42.651354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:42.656674 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ef630-ae22-430d-a9c3-23d9a254e5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:42.657614 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ef630-ae22-430d-a9c3-23d9a254e5b6 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:42.658039 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ef630-ae22-430d-a9c3-23d9a254e5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:42.658452 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ef630-ae22-430d-a9c3-23d9a254e5b6 None admin] PATH_INFO: `/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:42.659885 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ef630-ae22-430d-a9c3-23d9a254e5b6 None admin] RBAC: Authorizing `identity:get_project(project_id=e55f1cc9f9c9481698d6962c7af7fbfe)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:42.661610 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ef630-ae22-430d-a9c3-23d9a254e5b6 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:42.663496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 44/93] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:36:42 2022] GET /identity/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:42.664424 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ef630-ae22-430d-a9c3-23d9a254e5b6 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:42.773170 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7facb568-4423-4335-96ae-cd7d517f86f6 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:42.805106 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ef630-ae22-430d-a9c3-23d9a254e5b6 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:42.840115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cecf212-c4f9-40f6-aaa9-6439ab4dad1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:42.840720 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecf212-c4f9-40f6-aaa9-6439ab4dad1d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:42.840951 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecf212-c4f9-40f6-aaa9-6439ab4dad1d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:42.841233 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecf212-c4f9-40f6-aaa9-6439ab4dad1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:42.841991 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cecf212-c4f9-40f6-aaa9-6439ab4dad1d None admin] RBAC: Adding query filter params (role.id=b7a48a25b0cd479d85fe5395d64c458e, scope.project.id=e55f1cc9f9c9481698d6962c7af7fbfe, user.id=748c27d9ec1c4f29912ec71cb6f8863c) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:42.842311 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cecf212-c4f9-40f6-aaa9-6439ab4dad1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:42.845496 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cecf212-c4f9-40f6-aaa9-6439ab4dad1d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:42.851486 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82df946-1d2a-4faa-8c86-3b72c1541d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:42.852394 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82df946-1d2a-4faa-8c86-3b72c1541d6b None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:42.853021 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 50/94] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:36:42 2022] GET /identity/v3/role_assignments?user.id=748c27d9ec1c4f29912ec71cb6f8863c&scope.project.id=e55f1cc9f9c9481698d6962c7af7fbfe&role.id=b7a48a25b0cd479d85fe5395d64c458e => generated 259 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:42.853310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82df946-1d2a-4faa-8c86-3b72c1541d6b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:42.853729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82df946-1d2a-4faa-8c86-3b72c1541d6b None admin] PATH_INFO: `/v3/projects/ead3f51f90b34ba6852874282e70d280` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:42.853838 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cecf212-c4f9-40f6-aaa9-6439ab4dad1d None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:42.859936 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82df946-1d2a-4faa-8c86-3b72c1541d6b None admin] RBAC: Authorizing `identity:get_project(project_id=ead3f51f90b34ba6852874282e70d280)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:42.861553 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82df946-1d2a-4faa-8c86-3b72c1541d6b None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:42.871311 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 45/95] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:36:42 2022] GET /identity/v3/projects/ead3f51f90b34ba6852874282e70d280 => generated 312 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:42.873107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82df946-1d2a-4faa-8c86-3b72c1541d6b None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:42.923345 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:43.005129 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:43.045410 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cecf212-c4f9-40f6-aaa9-6439ab4dad1d None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:43.083559 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82df946-1d2a-4faa-8c86-3b72c1541d6b None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:43.098434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb58c91-6f45-495b-be7b-6298e384bb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:43.099442 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb58c91-6f45-495b-be7b-6298e384bb9d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:43.099950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb58c91-6f45-495b-be7b-6298e384bb9d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:43.100412 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb58c91-6f45-495b-be7b-6298e384bb9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:43.101368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb58c91-6f45-495b-be7b-6298e384bb9d None admin] RBAC: Adding query filter params (role.id=6195dc21ae4749daa31797d2c42e050e, scope.project.id=e55f1cc9f9c9481698d6962c7af7fbfe, user.id=748c27d9ec1c4f29912ec71cb6f8863c) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:43.101882 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb58c91-6f45-495b-be7b-6298e384bb9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:43.103590 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb58c91-6f45-495b-be7b-6298e384bb9d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:43.110035 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 51/96] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:36:42 2022] GET /identity/v3/role_assignments?user.id=748c27d9ec1c4f29912ec71cb6f8863c&scope.project.id=e55f1cc9f9c9481698d6962c7af7fbfe&role.id=6195dc21ae4749daa31797d2c42e050e => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:43.125062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb23f0-15b9-4f19-abf8-5433daaae0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:43.125792 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb23f0-15b9-4f19-abf8-5433daaae0ba None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:43.126078 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb23f0-15b9-4f19-abf8-5433daaae0ba None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:43.126358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb23f0-15b9-4f19-abf8-5433daaae0ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:43.127195 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb23f0-15b9-4f19-abf8-5433daaae0ba None admin] RBAC: Adding query filter params (role.id=b7a48a25b0cd479d85fe5395d64c458e, scope.project.id=ead3f51f90b34ba6852874282e70d280, user.id=748c27d9ec1c4f29912ec71cb6f8863c) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:43.127516 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb23f0-15b9-4f19-abf8-5433daaae0ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:43.129069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb23f0-15b9-4f19-abf8-5433daaae0ba None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:43.136304 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 46/97] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:36:42 2022] GET /identity/v3/role_assignments?user.id=748c27d9ec1c4f29912ec71cb6f8863c&scope.project.id=ead3f51f90b34ba6852874282e70d280&role.id=b7a48a25b0cd479d85fe5395d64c458e => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:44.394978 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab2789-d193-4cb1-982f-476d152dd8d8 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:44.395784 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab2789-d193-4cb1-982f-476d152dd8d8 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:44.395915 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab2789-d193-4cb1-982f-476d152dd8d8 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:44.398634 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 52/98] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:44.409322 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fda683-e255-4c2a-857f-b14387a4fa0e None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:44.410769 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fda683-e255-4c2a-857f-b14387a4fa0e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:44.411563 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fda683-e255-4c2a-857f-b14387a4fa0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:44.444013 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-a6fda683-e255-4c2a-857f-b14387a4fa0e None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:44.446286 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a6fda683-e255-4c2a-857f-b14387a4fa0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:44.504438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829a22a-a36f-4608-b980-d51f6633af7b None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:44.505075 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829a22a-a36f-4608-b980-d51f6633af7b None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:44.505593 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829a22a-a36f-4608-b980-d51f6633af7b None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:44.507806 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 53/99] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:44.515353 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa7412-5cbe-4ce1-9a28-72553b01d288 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:44.516036 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa7412-5cbe-4ce1-9a28-72553b01d288 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:44.516535 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa7412-5cbe-4ce1-9a28-72553b01d288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:44.547406 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-33aa7412-5cbe-4ce1-9a28-72553b01d288 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:44.549195 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-33aa7412-5cbe-4ce1-9a28-72553b01d288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:44.564396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 47/100] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:36:44.644850 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 54/101] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 08:36:44.949373 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fda683-e255-4c2a-857f-b14387a4fa0e None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:44.970166 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:45.071933 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fda683-e255-4c2a-857f-b14387a4fa0e None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:45.113890 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83bccf-d02f-416d-abbf-cc11edd255b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:45.114579 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83bccf-d02f-416d-abbf-cc11edd255b6 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:45.114858 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83bccf-d02f-416d-abbf-cc11edd255b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:45.115159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83bccf-d02f-416d-abbf-cc11edd255b6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:45.126439 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da83bccf-d02f-416d-abbf-cc11edd255b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:36:45.131328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-da83bccf-d02f-416d-abbf-cc11edd255b6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 08:36:45.131328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:36:45.131328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:36:45.131328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:36:45.131328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:36:45.131328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:36:45.131328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:36:45.131328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:36:45.131328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:36:45.131328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:36:45.131328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:36:45.131328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:36:45.131328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:36:45.131328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:36:45.131328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:36:45.131328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:36:45.132225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:36:45.132225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:36:45.132225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:36:45.132225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:36:45.132225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:36:45.132225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:36:45.132225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:36:45.132225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:36:45.132225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:36:45.132225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:36:45.132225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:36:45.132225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:36:45.132225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:36:45.132225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:36:45.132225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:36:45.132225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:36:45.133027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:36:45.133027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:36:45.133027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:36:45.133027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:36:45.133027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:36:45.133027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:36:45.133027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:36:45.133027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:36:45.133027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:36:45.133027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:36:45.133027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 08:36:45.133027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:36:45.133027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 48/102] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:36:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:36:45.138588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa7412-5cbe-4ce1-9a28-72553b01d288 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:45.251909 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:45.285137 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa7412-5cbe-4ce1-9a28-72553b01d288 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:45.340753 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae3a71-1a48-4b0b-9daa-7b87ec5f9684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:45.341411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae3a71-1a48-4b0b-9daa-7b87ec5f9684 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:45.341699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae3a71-1a48-4b0b-9daa-7b87ec5f9684 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:45.341981 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae3a71-1a48-4b0b-9daa-7b87ec5f9684 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:45.342679 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae3a71-1a48-4b0b-9daa-7b87ec5f9684 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:45.343017 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae3a71-1a48-4b0b-9daa-7b87ec5f9684 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:45.344681 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae3a71-1a48-4b0b-9daa-7b87ec5f9684 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:45.348019 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83bccf-d02f-416d-abbf-cc11edd255b6 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:45.360536 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 55/103] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:36:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:45.366834 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae3a71-1a48-4b0b-9daa-7b87ec5f9684 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:45.507441 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:45.543085 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83bccf-d02f-416d-abbf-cc11edd255b6 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:45.607758 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ade773-db0a-4e4f-949b-fcca45597ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:45.609041 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ade773-db0a-4e4f-949b-fcca45597ff0 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:45.609560 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ade773-db0a-4e4f-949b-fcca45597ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:45.610031 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ade773-db0a-4e4f-949b-fcca45597ff0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:45.613777 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae3a71-1a48-4b0b-9daa-7b87ec5f9684 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:45.615063 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ade773-db0a-4e4f-949b-fcca45597ff0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:45.617255 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ade773-db0a-4e4f-949b-fcca45597ff0 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:45.619280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 49/104] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:36:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:36:45.626924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ade773-db0a-4e4f-949b-fcca45597ff0 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:45.712269 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f94f83-1faf-43f2-99e9-ff89e9399a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:45.712955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f94f83-1faf-43f2-99e9-ff89e9399a96 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:45.713246 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f94f83-1faf-43f2-99e9-ff89e9399a96 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:45.713530 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f94f83-1faf-43f2-99e9-ff89e9399a96 None admin] PATH_INFO: `/v3/users/e0fac339988e453e8f6be2c7ba535863` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:45.731718 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:45.755532 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f94f83-1faf-43f2-99e9-ff89e9399a96 None admin] RBAC: Authorizing `identity:get_user(user_id=e0fac339988e453e8f6be2c7ba535863)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:45.757541 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f94f83-1faf-43f2-99e9-ff89e9399a96 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:45.761625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 56/105] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:36:45 2022] GET /identity/v3/users/e0fac339988e453e8f6be2c7ba535863 => generated 289 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:36:45.770168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f94f83-1faf-43f2-99e9-ff89e9399a96 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:45.783642 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ade773-db0a-4e4f-949b-fcca45597ff0 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:45.833436 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f71bc-1d3b-4f77-be7f-1c0ddfb62fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:45.834452 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f71bc-1d3b-4f77-be7f-1c0ddfb62fb1 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:45.835057 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f71bc-1d3b-4f77-be7f-1c0ddfb62fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:45.835547 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f71bc-1d3b-4f77-be7f-1c0ddfb62fb1 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:45.836412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f71bc-1d3b-4f77-be7f-1c0ddfb62fb1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:45.838096 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f71bc-1d3b-4f77-be7f-1c0ddfb62fb1 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:45.877647 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:45.908599 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 50/106] 10.222.0.71 () {60 vars in 1141 bytes} [Fri Aug 26 08:36:45 2022] POST /identity/v3/projects => generated 302 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:36:45.910353 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b7a93-e8d3-4552-8a5c-10aee8e82045 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:45.910871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b7a93-e8d3-4552-8a5c-10aee8e82045 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:45.911364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b7a93-e8d3-4552-8a5c-10aee8e82045 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:45.913387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 51/107] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:45.919599 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e38767-ba52-4d27-bba4-a89a96ac3d35 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:45.920268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e38767-ba52-4d27-bba4-a89a96ac3d35 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:45.920950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e38767-ba52-4d27-bba4-a89a96ac3d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:45.955025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-07e38767-ba52-4d27-bba4-a89a96ac3d35 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:45.957808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-07e38767-ba52-4d27-bba4-a89a96ac3d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:45.993906 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f94f83-1faf-43f2-99e9-ff89e9399a96 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:46.075199 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89de59-9f98-4778-951d-a4ebb954c187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:46.075789 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89de59-9f98-4778-951d-a4ebb954c187 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:46.076078 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89de59-9f98-4778-951d-a4ebb954c187 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:46.076407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89de59-9f98-4778-951d-a4ebb954c187 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:46.103784 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 52/108] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:36:46.179087 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:46.188657 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89de59-9f98-4778-951d-a4ebb954c187 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:46.190218 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89de59-9f98-4778-951d-a4ebb954c187 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:46.315363 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 57/109] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:36:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:36:46.320885 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e38767-ba52-4d27-bba4-a89a96ac3d35 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:46.462582 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e38767-ba52-4d27-bba4-a89a96ac3d35 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:46.486775 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89de59-9f98-4778-951d-a4ebb954c187 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:46.494559 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:46.509053 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfc33d-342b-46fa-9649-5d4782afe208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:46.509850 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfc33d-342b-46fa-9649-5d4782afe208 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:46.510019 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfc33d-342b-46fa-9649-5d4782afe208 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:46.510290 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfc33d-342b-46fa-9649-5d4782afe208 None admin] PATH_INFO: `/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:46.511624 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdfc33d-342b-46fa-9649-5d4782afe208 None admin] RBAC: Authorizing `identity:get_project(project_id=e55f1cc9f9c9481698d6962c7af7fbfe)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:46.513229 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdfc33d-342b-46fa-9649-5d4782afe208 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:46.515322 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 53/110] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:36:46 2022] GET /identity/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:46.521288 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfc33d-342b-46fa-9649-5d4782afe208 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:46.655109 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:46.662720 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89de59-9f98-4778-951d-a4ebb954c187 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:46.727815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c5ede-a4d8-4a8e-a650-ba5a1ed25032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:46.728718 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c5ede-a4d8-4a8e-a650-ba5a1ed25032 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:46.729051 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c5ede-a4d8-4a8e-a650-ba5a1ed25032 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:46.729479 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c5ede-a4d8-4a8e-a650-ba5a1ed25032 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:46.736528 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfc33d-342b-46fa-9649-5d4782afe208 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:46.747178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27c5ede-a4d8-4a8e-a650-ba5a1ed25032 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:36:46.753294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-d27c5ede-a4d8-4a8e-a650-ba5a1ed25032 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:36:46.753294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:36:46.753294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:36:46.753294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:36:46.753294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:36:46.753294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:36:46.753294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:36:46.753294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:36:46.753294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:36:46.753294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:36:46.753294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:36:46.753294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:36:46.753294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:36:46.753294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:36:46.753294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:36:46.753294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:36:46.754200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:36:46.754200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:36:46.754200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:36:46.754200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:36:46.754200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:36:46.754200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:36:46.754200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:36:46.754200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:36:46.754200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:36:46.754200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:36:46.754200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:36:46.754200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:36:46.754200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:36:46.754200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:36:46.754200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:36:46.754200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:36:46.755142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:36:46.755142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:36:46.755142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:36:46.755142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:36:46.755142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:36:46.755142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:36:46.755142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:36:46.755142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:36:46.755142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:36:46.755142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:36:46.755142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:36:46.755142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:36:46.756708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 58/111] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:36:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:36:46.760442 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a633f7e-d0ec-409a-aa34-73cb6421e322 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:46.760442 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a633f7e-d0ec-409a-aa34-73cb6421e322 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:46.760442 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a633f7e-d0ec-409a-aa34-73cb6421e322 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:46.761043 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 59/112] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:46.762564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6ad1f-3be7-4b95-98e4-a02dc92dbea1 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:46.762873 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6ad1f-3be7-4b95-98e4-a02dc92dbea1 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:46.763178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6ad1f-3be7-4b95-98e4-a02dc92dbea1 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:46.765068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 60/113] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:46.765844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6ad1f-3be7-4b95-98e4-a02dc92dbea1 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:46.789496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0851bc93-ed1b-42c1-8636-72e738f722f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:46.790239 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851bc93-ed1b-42c1-8636-72e738f722f3 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:46.790608 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851bc93-ed1b-42c1-8636-72e738f722f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:46.790982 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851bc93-ed1b-42c1-8636-72e738f722f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:46.791826 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0851bc93-ed1b-42c1-8636-72e738f722f3 None admin] RBAC: Adding query filter params (role.id=0f660bd0ff4c4beda97ed7c24851d2d4, scope.project.id=e55f1cc9f9c9481698d6962c7af7fbfe, user.id=e0fac339988e453e8f6be2c7ba535863) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:46.792204 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0851bc93-ed1b-42c1-8636-72e738f722f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:46.793754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0851bc93-ed1b-42c1-8636-72e738f722f3 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:46.800213 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 54/114] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:36:46 2022] GET /identity/v3/role_assignments?user.id=e0fac339988e453e8f6be2c7ba535863&scope.project.id=e55f1cc9f9c9481698d6962c7af7fbfe&role.id=0f660bd0ff4c4beda97ed7c24851d2d4 => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:36:46.801428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0072a2-db2f-45b8-9ea4-e40c6558145e None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:46.801964 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0072a2-db2f-45b8-9ea4-e40c6558145e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:46.802357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0072a2-db2f-45b8-9ea4-e40c6558145e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:46.834831 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-4f0072a2-db2f-45b8-9ea4-e40c6558145e None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:46.840348 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4f0072a2-db2f-45b8-9ea4-e40c6558145e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:46.983097 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:47.000050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6ad1f-3be7-4b95-98e4-a02dc92dbea1 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:47.025006 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 55/115] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:36:47.026797 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8743f97-76fc-47db-8b4f-ac3064aa2226 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:47.030087 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8743f97-76fc-47db-8b4f-ac3064aa2226 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:47.030715 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8743f97-76fc-47db-8b4f-ac3064aa2226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:47.060373 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-e8743f97-76fc-47db-8b4f-ac3064aa2226 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:47.061714 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e8743f97-76fc-47db-8b4f-ac3064aa2226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:47.078287 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8171825-e82c-42eb-8cf0-00abafc28ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:47.079063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8171825-e82c-42eb-8cf0-00abafc28ef2 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:47.079355 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8171825-e82c-42eb-8cf0-00abafc28ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:47.079639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8171825-e82c-42eb-8cf0-00abafc28ef2 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:47.080340 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8171825-e82c-42eb-8cf0-00abafc28ef2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:47.080652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8171825-e82c-42eb-8cf0-00abafc28ef2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:47.082076 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8171825-e82c-42eb-8cf0-00abafc28ef2 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:47.092588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 61/116] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:36:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:47.099049 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96170a9b-bd33-46f0-8be3-cb5d51ba9e62 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:47.099386 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96170a9b-bd33-46f0-8be3-cb5d51ba9e62 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:47.099626 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96170a9b-bd33-46f0-8be3-cb5d51ba9e62 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:47.101697 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 62/117] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:47.104265 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96170a9b-bd33-46f0-8be3-cb5d51ba9e62 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:47.130450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:47.152722 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 56/118] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:36:47.155762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd88d952-7a9e-47da-bb96-d9453fb4a91c None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:47.156933 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd88d952-7a9e-47da-bb96-d9453fb4a91c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:47.158054 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd88d952-7a9e-47da-bb96-d9453fb4a91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:47.195893 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-bd88d952-7a9e-47da-bb96-d9453fb4a91c None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:47.198059 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-bd88d952-7a9e-47da-bb96-d9453fb4a91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:47.323687 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96170a9b-bd33-46f0-8be3-cb5d51ba9e62 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:47.389529 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 57/119] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:36:47.411607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd88d952-7a9e-47da-bb96-d9453fb4a91c None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:47.415065 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c970fd-f5b3-4ec5-af5c-687cf823b33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:47.416881 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c970fd-f5b3-4ec5-af5c-687cf823b33c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:47.417318 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c970fd-f5b3-4ec5-af5c-687cf823b33c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:47.417733 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c970fd-f5b3-4ec5-af5c-687cf823b33c None admin] PATH_INFO: `/v3/users/668b82af2c1345cca572ef92fff86ff2` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:47.421940 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c970fd-f5b3-4ec5-af5c-687cf823b33c None admin] RBAC: Authorizing `identity:get_user(user_id=668b82af2c1345cca572ef92fff86ff2)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:47.424170 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c970fd-f5b3-4ec5-af5c-687cf823b33c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:47.426732 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 63/120] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:36:47 2022] GET /identity/v3/users/668b82af2c1345cca572ef92fff86ff2 => generated 247 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:47.432603 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c970fd-f5b3-4ec5-af5c-687cf823b33c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:47.475293 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:47.527837 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd88d952-7a9e-47da-bb96-d9453fb4a91c None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:47.578066 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251c9f3-122e-4992-8854-a7ebf21c8717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:47.578638 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251c9f3-122e-4992-8854-a7ebf21c8717 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:47.578993 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251c9f3-122e-4992-8854-a7ebf21c8717 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:47.579272 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251c9f3-122e-4992-8854-a7ebf21c8717 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:47.601339 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c251c9f3-122e-4992-8854-a7ebf21c8717 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:36:47.611962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-c251c9f3-122e-4992-8854-a7ebf21c8717 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 08:36:47.611962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:36:47.611962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:36:47.611962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:36:47.611962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:36:47.611962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:36:47.611962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:36:47.611962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:36:47.611962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:36:47.611962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:36:47.611962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:36:47.611962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:36:47.611962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:36:47.611962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:36:47.611962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:36:47.611962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:36:47.612886 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:36:47.612886 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:36:47.612886 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:36:47.612886 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:36:47.612886 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:36:47.612886 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:36:47.612886 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:36:47.612886 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:36:47.612886 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:36:47.612886 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:36:47.612886 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:36:47.612886 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:36:47.612886 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:36:47.612886 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:36:47.612886 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:36:47.612886 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:36:47.614023 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:36:47.614023 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:36:47.614023 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:36:47.614023 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:36:47.614023 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:36:47.614023 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:36:47.614023 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:36:47.614023 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:36:47.614023 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:36:47.614023 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:36:47.614023 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 08:36:47.614023 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:36:47.614023 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 58/121] 10.222.0.71 () {56 vars in 1114 bytes} [Fri Aug 26 08:36:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:36:47.614671 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251c9f3-122e-4992-8854-a7ebf21c8717 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:47.637907 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:47.657171 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c970fd-f5b3-4ec5-af5c-687cf823b33c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:47.698504 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9432c2e4-3191-47d5-a4ce-b37c282cbd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:47.699414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432c2e4-3191-47d5-a4ce-b37c282cbd5c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:47.699710 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432c2e4-3191-47d5-a4ce-b37c282cbd5c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:47.700048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9432c2e4-3191-47d5-a4ce-b37c282cbd5c None admin] PATH_INFO: `/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:47.702241 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9432c2e4-3191-47d5-a4ce-b37c282cbd5c None admin] RBAC: Authorizing `identity:get_project(project_id=e55f1cc9f9c9481698d6962c7af7fbfe)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:47.703776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9432c2e4-3191-47d5-a4ce-b37c282cbd5c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:47.706084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 64/122] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:36:47 2022] GET /identity/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe => generated 298 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:36:47.706914 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9432c2e4-3191-47d5-a4ce-b37c282cbd5c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:47.777189 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251c9f3-122e-4992-8854-a7ebf21c8717 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:47.786084 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:47.816320 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952f3cf-54e0-4359-807e-bb86ea63fead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:47.817105 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952f3cf-54e0-4359-807e-bb86ea63fead None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:47.817553 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952f3cf-54e0-4359-807e-bb86ea63fead None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:47.817852 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952f3cf-54e0-4359-807e-bb86ea63fead None admin] PATH_INFO: `/v3/roles/member` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:47.833507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952f3cf-54e0-4359-807e-bb86ea63fead None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:36:47.840844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-4952f3cf-54e0-4359-807e-bb86ea63fead None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:36:47.840844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:36:47.840844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:36:47.840844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:36:47.840844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:36:47.840844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:36:47.840844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:36:47.840844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:36:47.840844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:36:47.840844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:36:47.840844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:36:47.840844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:36:47.840844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:36:47.840844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:36:47.840844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:36:47.840844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:36:47.842920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:36:47.842920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:36:47.842920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:36:47.842920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:36:47.842920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:36:47.842920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:36:47.842920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:36:47.842920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:36:47.842920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:36:47.842920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:36:47.842920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:36:47.842920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:36:47.842920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:36:47.842920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:36:47.842920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:36:47.842920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:36:47.844901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:36:47.844901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:36:47.844901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:36:47.844901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:36:47.844901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:36:47.844901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:36:47.844901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:36:47.844901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:36:47.844901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:36:47.844901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:36:47.844901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:36:47.844901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:36:47.844901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 59/123] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:36:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:36:47.844901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952f3cf-54e0-4359-807e-bb86ea63fead None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:47.846547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9432c2e4-3191-47d5-a4ce-b37c282cbd5c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:47.894498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe7991-e174-45a5-8358-4899f8fa3f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:47.895156 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe7991-e174-45a5-8358-4899f8fa3f79 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:47.895441 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe7991-e174-45a5-8358-4899f8fa3f79 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:47.895718 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe7991-e174-45a5-8358-4899f8fa3f79 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:47.896676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe7991-e174-45a5-8358-4899f8fa3f79 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:47.896988 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe7991-e174-45a5-8358-4899f8fa3f79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:47.899412 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe7991-e174-45a5-8358-4899f8fa3f79 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:47.911078 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 65/124] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:36:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:47.916688 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe7991-e174-45a5-8358-4899f8fa3f79 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:47.936149 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:47.986031 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952f3cf-54e0-4359-807e-bb86ea63fead None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:48.023874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6c2fe-da69-4ecc-97e8-469a1742c53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:48.024477 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6c2fe-da69-4ecc-97e8-469a1742c53b None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:48.024758 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6c2fe-da69-4ecc-97e8-469a1742c53b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:48.025031 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6c2fe-da69-4ecc-97e8-469a1742c53b None admin] PATH_INFO: `/v3/roles/member` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:48.037224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b6c2fe-da69-4ecc-97e8-469a1742c53b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:36:48.042432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-c1b6c2fe-da69-4ecc-97e8-469a1742c53b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:36:48.042432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:36:48.042432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:36:48.042432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:36:48.042432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:36:48.042432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:36:48.042432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:36:48.042432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:36:48.042432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:36:48.042432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:36:48.042432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:36:48.042432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:36:48.042432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:36:48.042432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:36:48.042432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:36:48.042432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:36:48.043518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:36:48.043518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:36:48.043518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:36:48.043518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:36:48.043518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:36:48.043518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:36:48.043518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:36:48.043518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:36:48.043518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:36:48.043518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:36:48.043518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:36:48.043518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:36:48.043518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:36:48.043518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:36:48.043518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:36:48.043518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:36:48.044380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:36:48.044380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:36:48.044380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:36:48.044380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:36:48.044380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:36:48.044380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:36:48.044380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:36:48.044380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:36:48.044380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:36:48.044380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:36:48.044380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:36:48.044380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:36:48.044380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 60/125] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:36:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:36:48.044380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6c2fe-da69-4ecc-97e8-469a1742c53b None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:48.045356 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe7991-e174-45a5-8358-4899f8fa3f79 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:48.075735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:48.092526 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a484a7-4b64-46bb-a1ad-ff9f301655ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:48.093448 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a484a7-4b64-46bb-a1ad-ff9f301655ce None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:48.093864 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a484a7-4b64-46bb-a1ad-ff9f301655ce None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:48.094281 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a484a7-4b64-46bb-a1ad-ff9f301655ce None admin] PATH_INFO: `/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe/users/668b82af2c1345cca572ef92fff86ff2/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:48.097422 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a484a7-4b64-46bb-a1ad-ff9f301655ce None admin] RBAC: Authorizing `identity:create_grant(project_id=e55f1cc9f9c9481698d6962c7af7fbfe, user_id=668b82af2c1345cca572ef92fff86ff2, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:48.099615 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a484a7-4b64-46bb-a1ad-ff9f301655ce None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:48.119755 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 66/126] 10.222.0.71 () {58 vars in 1439 bytes} [Fri Aug 26 08:36:47 2022] PUT /identity/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe/users/668b82af2c1345cca572ef92fff86ff2/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:36:48.121041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a484a7-4b64-46bb-a1ad-ff9f301655ce None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:48.197574 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6c2fe-da69-4ecc-97e8-469a1742c53b None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:48.220310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:48.252910 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a484a7-4b64-46bb-a1ad-ff9f301655ce None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:48.272599 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2afef-0b61-4266-a7c1-a4132af206d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:48.273183 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2afef-0b61-4266-a7c1-a4132af206d4 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:48.273464 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2afef-0b61-4266-a7c1-a4132af206d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:48.273751 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2afef-0b61-4266-a7c1-a4132af206d4 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:48.274452 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b2afef-0b61-4266-a7c1-a4132af206d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:48.274755 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b2afef-0b61-4266-a7c1-a4132af206d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:48.276234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b2afef-0b61-4266-a7c1-a4132af206d4 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:48.287614 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 61/127] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:36:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:48.288529 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2afef-0b61-4266-a7c1-a4132af206d4 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:48.302145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e1040-8fa6-41cd-be2b-418c543b1a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:48.302810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e1040-8fa6-41cd-be2b-418c543b1a4a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:48.303125 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e1040-8fa6-41cd-be2b-418c543b1a4a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:48.303402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e1040-8fa6-41cd-be2b-418c543b1a4a None admin] PATH_INFO: `/v3/users/748c27d9ec1c4f29912ec71cb6f8863c` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:48.304592 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e1040-8fa6-41cd-be2b-418c543b1a4a None admin] RBAC: Authorizing `identity:get_user(user_id=748c27d9ec1c4f29912ec71cb6f8863c)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:48.306063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e1040-8fa6-41cd-be2b-418c543b1a4a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:48.307931 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 67/128] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:36:48 2022] GET /identity/v3/users/748c27d9ec1c4f29912ec71cb6f8863c => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:48.308494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e1040-8fa6-41cd-be2b-418c543b1a4a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:48.406448 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:48.420913 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2afef-0b61-4266-a7c1-a4132af206d4 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:48.430064 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e1040-8fa6-41cd-be2b-418c543b1a4a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:48.458011 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53112aa8-2897-4b34-b96e-14ebb9d83d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:48.458934 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53112aa8-2897-4b34-b96e-14ebb9d83d6a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:48.459307 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53112aa8-2897-4b34-b96e-14ebb9d83d6a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:48.459608 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53112aa8-2897-4b34-b96e-14ebb9d83d6a None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:48.460485 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53112aa8-2897-4b34-b96e-14ebb9d83d6a None admin] RBAC: Adding query filter params (name=member) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:48.461100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53112aa8-2897-4b34-b96e-14ebb9d83d6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:48.462849 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53112aa8-2897-4b34-b96e-14ebb9d83d6a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:48.465656 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b733cc-7df1-4799-b40f-e8ef7919ed73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:48.466384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b733cc-7df1-4799-b40f-e8ef7919ed73 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:48.466720 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b733cc-7df1-4799-b40f-e8ef7919ed73 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:48.467103 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b733cc-7df1-4799-b40f-e8ef7919ed73 None admin] PATH_INFO: `/v3/users/748c27d9ec1c4f29912ec71cb6f8863c` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:48.468329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b733cc-7df1-4799-b40f-e8ef7919ed73 None admin] RBAC: Authorizing `identity:get_user(user_id=748c27d9ec1c4f29912ec71cb6f8863c)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:48.470021 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b733cc-7df1-4799-b40f-e8ef7919ed73 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:48.472071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 68/129] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:36:48 2022] GET /identity/v3/users/748c27d9ec1c4f29912ec71cb6f8863c => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:48.472602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 62/130] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:36:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:48.473343 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b733cc-7df1-4799-b40f-e8ef7919ed73 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:48.478302 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53112aa8-2897-4b34-b96e-14ebb9d83d6a None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:48.511280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:48.617377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b733cc-7df1-4799-b40f-e8ef7919ed73 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:48.650600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53112aa8-2897-4b34-b96e-14ebb9d83d6a None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:48.690373 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fdcbc5-3dd8-444a-bb31-966ba05cc95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:48.691398 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fdcbc5-3dd8-444a-bb31-966ba05cc95b None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:48.691852 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fdcbc5-3dd8-444a-bb31-966ba05cc95b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:48.692312 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fdcbc5-3dd8-444a-bb31-966ba05cc95b None admin] PATH_INFO: `/v3/users/748c27d9ec1c4f29912ec71cb6f8863c` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:48.693735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fdcbc5-3dd8-444a-bb31-966ba05cc95b None admin] RBAC: Authorizing `identity:get_user(user_id=748c27d9ec1c4f29912ec71cb6f8863c)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:48.695445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fdcbc5-3dd8-444a-bb31-966ba05cc95b None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:48.696914 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be2ed0-f100-4ac9-90bb-911469a7d1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:48.697596 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be2ed0-f100-4ac9-90bb-911469a7d1d8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:48.697877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be2ed0-f100-4ac9-90bb-911469a7d1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:48.698150 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be2ed0-f100-4ac9-90bb-911469a7d1d8 None admin] PATH_INFO: `/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:48.698564 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 63/131] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:36:48 2022] GET /identity/v3/users/748c27d9ec1c4f29912ec71cb6f8863c => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:36:48.699619 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fdcbc5-3dd8-444a-bb31-966ba05cc95b None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:48.703708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be2ed0-f100-4ac9-90bb-911469a7d1d8 None admin] RBAC: Authorizing `identity:get_project(project_id=e55f1cc9f9c9481698d6962c7af7fbfe)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:48.705283 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be2ed0-f100-4ac9-90bb-911469a7d1d8 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:48.707207 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 69/132] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:36:48 2022] GET /identity/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:48.707848 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be2ed0-f100-4ac9-90bb-911469a7d1d8 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:48.750915 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:48.799280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:48.818577 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be2ed0-f100-4ac9-90bb-911469a7d1d8 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:48.854795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034b259-cbbf-4f60-8ddd-a26b9b808abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:48.855521 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034b259-cbbf-4f60-8ddd-a26b9b808abd None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:48.855805 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034b259-cbbf-4f60-8ddd-a26b9b808abd None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:48.856083 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8034b259-cbbf-4f60-8ddd-a26b9b808abd None admin] PATH_INFO: `/v3/projects/ead3f51f90b34ba6852874282e70d280` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:48.857284 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8034b259-cbbf-4f60-8ddd-a26b9b808abd None admin] RBAC: Authorizing `identity:get_project(project_id=ead3f51f90b34ba6852874282e70d280)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:48.858752 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8034b259-cbbf-4f60-8ddd-a26b9b808abd None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:48.860562 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 70/133] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:36:48 2022] GET /identity/v3/projects/ead3f51f90b34ba6852874282e70d280 => generated 312 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:48.861373 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034b259-cbbf-4f60-8ddd-a26b9b808abd None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:48.873193 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fdcbc5-3dd8-444a-bb31-966ba05cc95b None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:48.913302 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46a776-9c67-451a-aefc-f5c7335f37a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:48.914473 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46a776-9c67-451a-aefc-f5c7335f37a9 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:48.915172 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46a776-9c67-451a-aefc-f5c7335f37a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:48.915675 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46a776-9c67-451a-aefc-f5c7335f37a9 None admin] PATH_INFO: `/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:48.917065 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46a776-9c67-451a-aefc-f5c7335f37a9 None admin] RBAC: Authorizing `identity:get_project(project_id=e55f1cc9f9c9481698d6962c7af7fbfe)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:48.919157 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46a776-9c67-451a-aefc-f5c7335f37a9 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:48.921327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 64/134] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:36:48 2022] GET /identity/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:48.923105 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c959da2-9d1e-4ee2-a653-adbef0252634 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:48.923704 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c959da2-9d1e-4ee2-a653-adbef0252634 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:48.924237 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c959da2-9d1e-4ee2-a653-adbef0252634 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:48.925938 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 65/135] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:48.931424 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c959da2-9d1e-4ee2-a653-adbef0252634 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:48.969052 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8034b259-cbbf-4f60-8ddd-a26b9b808abd None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:49.006851 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a58ce-5d80-46c6-9681-21abbb2b0896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:49.008032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a58ce-5d80-46c6-9681-21abbb2b0896 None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:49.008375 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a58ce-5d80-46c6-9681-21abbb2b0896 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:49.008707 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a58ce-5d80-46c6-9681-21abbb2b0896 None admin] PATH_INFO: `/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe/users/748c27d9ec1c4f29912ec71cb6f8863c/roles/6195dc21ae4749daa31797d2c42e050e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:49.011039 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a58ce-5d80-46c6-9681-21abbb2b0896 None admin] RBAC: Authorizing `identity:create_grant(project_id=e55f1cc9f9c9481698d6962c7af7fbfe, user_id=748c27d9ec1c4f29912ec71cb6f8863c, role_id=6195dc21ae4749daa31797d2c42e050e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:49.012721 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a58ce-5d80-46c6-9681-21abbb2b0896 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:49.030384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 71/136] 10.222.0.71 () {58 vars in 1439 bytes} [Fri Aug 26 08:36:48 2022] PUT /identity/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe/users/748c27d9ec1c4f29912ec71cb6f8863c/roles/6195dc21ae4749daa31797d2c42e050e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:36:49.038603 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a58ce-5d80-46c6-9681-21abbb2b0896 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:49.063505 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:49.068359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:49.093124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c959da2-9d1e-4ee2-a653-adbef0252634 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:49.167869 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83bb8a6-fd6a-4efc-902a-c2dd082fca75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:49.169432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83bb8a6-fd6a-4efc-902a-c2dd082fca75 None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:49.169717 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83bb8a6-fd6a-4efc-902a-c2dd082fca75 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:49.169998 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83bb8a6-fd6a-4efc-902a-c2dd082fca75 None admin] PATH_INFO: `/v3/projects/ead3f51f90b34ba6852874282e70d280/users/748c27d9ec1c4f29912ec71cb6f8863c/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:49.175787 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83bb8a6-fd6a-4efc-902a-c2dd082fca75 None admin] RBAC: Authorizing `identity:create_grant(project_id=ead3f51f90b34ba6852874282e70d280, user_id=748c27d9ec1c4f29912ec71cb6f8863c, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:49.177333 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83bb8a6-fd6a-4efc-902a-c2dd082fca75 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:49.201710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 66/137] 10.222.0.71 () {58 vars in 1439 bytes} [Fri Aug 26 08:36:48 2022] PUT /identity/v3/projects/ead3f51f90b34ba6852874282e70d280/users/748c27d9ec1c4f29912ec71cb6f8863c/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:36:49.203602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41585c49-0006-4eac-a8dc-df636eabd80b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:49.204450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41585c49-0006-4eac-a8dc-df636eabd80b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:49.204742 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41585c49-0006-4eac-a8dc-df636eabd80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:49.241111 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-41585c49-0006-4eac-a8dc-df636eabd80b None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:49.242555 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-41585c49-0006-4eac-a8dc-df636eabd80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:49.249769 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a58ce-5d80-46c6-9681-21abbb2b0896 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:49.302252 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662ecb2-3a46-48a0-b813-9c00d70baf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:49.303613 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662ecb2-3a46-48a0-b813-9c00d70baf7c None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:49.304576 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662ecb2-3a46-48a0-b813-9c00d70baf7c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:49.305208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662ecb2-3a46-48a0-b813-9c00d70baf7c None admin] PATH_INFO: `/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe/users/748c27d9ec1c4f29912ec71cb6f8863c/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:49.308179 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d662ecb2-3a46-48a0-b813-9c00d70baf7c None admin] RBAC: Authorizing `identity:create_grant(project_id=e55f1cc9f9c9481698d6962c7af7fbfe, user_id=748c27d9ec1c4f29912ec71cb6f8863c, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:49.309911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d662ecb2-3a46-48a0-b813-9c00d70baf7c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:49.329806 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 72/138] 10.222.0.71 () {58 vars in 1439 bytes} [Fri Aug 26 08:36:49 2022] PUT /identity/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe/users/748c27d9ec1c4f29912ec71cb6f8863c/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:36:49.330859 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 67/139] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:36:49.374931 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801a774-15ba-43ef-811f-16c466a6c77e None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:49.374931 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801a774-15ba-43ef-811f-16c466a6c77e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:49.374931 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801a774-15ba-43ef-811f-16c466a6c77e None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:49.375657 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 73/140] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:49.382459 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26039181-89ba-44ea-be02-67b53a8911bd None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:49.382736 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26039181-89ba-44ea-be02-67b53a8911bd None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:49.383074 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26039181-89ba-44ea-be02-67b53a8911bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:49.416234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-26039181-89ba-44ea-be02-67b53a8911bd None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:49.417508 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-26039181-89ba-44ea-be02-67b53a8911bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:49.504249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 68/141] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 08:36:49.714957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4801a774-15ba-43ef-811f-16c466a6c77e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:49.807427 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26039181-89ba-44ea-be02-67b53a8911bd None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:49.807914 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:49.824193 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:49.861398 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4801a774-15ba-43ef-811f-16c466a6c77e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:49.907411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad44a75-2a44-45b6-a630-e4d8ae7c7e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:49.908082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad44a75-2a44-45b6-a630-e4d8ae7c7e83 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:49.908408 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad44a75-2a44-45b6-a630-e4d8ae7c7e83 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:49.908757 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad44a75-2a44-45b6-a630-e4d8ae7c7e83 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:49.910439 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad44a75-2a44-45b6-a630-e4d8ae7c7e83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:49.912183 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad44a75-2a44-45b6-a630-e4d8ae7c7e83 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:49.916044 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 74/142] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:36:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:49.927316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad44a75-2a44-45b6-a630-e4d8ae7c7e83 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:50.081819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26039181-89ba-44ea-be02-67b53a8911bd None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:50.088945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad44a75-2a44-45b6-a630-e4d8ae7c7e83 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:50.135346 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b37ffd-6a79-4a81-9517-42b153bb2057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:50.136101 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b37ffd-6a79-4a81-9517-42b153bb2057 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:50.136483 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b37ffd-6a79-4a81-9517-42b153bb2057 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:50.136863 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b37ffd-6a79-4a81-9517-42b153bb2057 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:50.137793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b37ffd-6a79-4a81-9517-42b153bb2057 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:50.140047 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b37ffd-6a79-4a81-9517-42b153bb2057 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:50.152301 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:50.156576 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ed1a0-d64f-433f-a98f-33e1a74ebe4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:50.157580 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ed1a0-d64f-433f-a98f-33e1a74ebe4e None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:50.157795 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ed1a0-d64f-433f-a98f-33e1a74ebe4e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:50.158084 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ed1a0-d64f-433f-a98f-33e1a74ebe4e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:50.173616 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 75/143] 10.222.0.71 () {60 vars in 1132 bytes} [Fri Aug 26 08:36:49 2022] POST /identity/v3/users => generated 283 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:36:50.185216 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ed1a0-d64f-433f-a98f-33e1a74ebe4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:36:50.190200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-a63ed1a0-d64f-433f-a98f-33e1a74ebe4e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 08:36:50.190200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:36:50.190200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:36:50.190200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:36:50.190200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:36:50.190200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:36:50.190200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:36:50.190200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:36:50.190200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:36:50.190200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:36:50.190200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:36:50.190200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:36:50.190200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:36:50.190200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:36:50.190200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:36:50.190200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:36:50.191191 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:36:50.191191 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:36:50.191191 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:36:50.191191 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:36:50.191191 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:36:50.191191 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:36:50.191191 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:36:50.191191 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:36:50.191191 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:36:50.191191 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:36:50.191191 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:36:50.191191 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:36:50.191191 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:36:50.191191 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:36:50.191191 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:36:50.191191 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:36:50.191951 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:36:50.191951 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:36:50.191951 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:36:50.191951 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:36:50.191951 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:36:50.191951 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:36:50.191951 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:36:50.191951 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:36:50.191951 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:36:50.191951 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:36:50.191951 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 08:36:50.191951 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:36:50.200142 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 69/144] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:36:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:36:50.206442 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b37ffd-6a79-4a81-9517-42b153bb2057 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:50.383637 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b37ffd-6a79-4a81-9517-42b153bb2057 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:50.426929 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ae767-ea8c-40ca-9fba-119ac15dd29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:50.427614 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ae767-ea8c-40ca-9fba-119ac15dd29e None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:50.427891 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ae767-ea8c-40ca-9fba-119ac15dd29e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:50.428163 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ae767-ea8c-40ca-9fba-119ac15dd29e None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:50.428901 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3ae767-ea8c-40ca-9fba-119ac15dd29e None admin] RBAC: Adding query filter params (name=reader) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:50.429356 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3ae767-ea8c-40ca-9fba-119ac15dd29e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:50.430809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3ae767-ea8c-40ca-9fba-119ac15dd29e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:50.442126 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 76/145] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:36:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:50.448355 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ed1a0-d64f-433f-a98f-33e1a74ebe4e None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:50.540966 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:50.587680 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ed1a0-d64f-433f-a98f-33e1a74ebe4e None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:50.646223 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211eb91-bec4-4f32-82d7-3941c7315eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:50.647322 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211eb91-bec4-4f32-82d7-3941c7315eac None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:50.648276 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211eb91-bec4-4f32-82d7-3941c7315eac None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:50.648575 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7211eb91-bec4-4f32-82d7-3941c7315eac None admin] PATH_INFO: `/v3/users/e0fac339988e453e8f6be2c7ba535863` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:50.650183 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7211eb91-bec4-4f32-82d7-3941c7315eac None admin] RBAC: Authorizing `identity:get_user(user_id=e0fac339988e453e8f6be2c7ba535863)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:50.651916 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7211eb91-bec4-4f32-82d7-3941c7315eac None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:50.653741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 70/146] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:36:50 2022] GET /identity/v3/users/e0fac339988e453e8f6be2c7ba535863 => generated 289 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:50.660158 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ae767-ea8c-40ca-9fba-119ac15dd29e None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:50.729916 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:50.791556 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ae767-ea8c-40ca-9fba-119ac15dd29e None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:50.834010 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d816e5-c93e-4fa3-b3ea-95455b3ad1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:50.834701 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d816e5-c93e-4fa3-b3ea-95455b3ad1ef None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:50.835050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d816e5-c93e-4fa3-b3ea-95455b3ad1ef None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:50.835358 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d816e5-c93e-4fa3-b3ea-95455b3ad1ef None admin] PATH_INFO: `/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:50.836760 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d816e5-c93e-4fa3-b3ea-95455b3ad1ef None admin] RBAC: Authorizing `identity:get_project(project_id=e55f1cc9f9c9481698d6962c7af7fbfe)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:50.838378 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d816e5-c93e-4fa3-b3ea-95455b3ad1ef None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:50.840315 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 77/147] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:36:50 2022] GET /identity/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:50.850509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211eb91-bec4-4f32-82d7-3941c7315eac None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:51.007852 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7211eb91-bec4-4f32-82d7-3941c7315eac None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:51.050639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b362b9-3dc9-4b6d-8969-ca99e9240439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:51.051387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b362b9-3dc9-4b6d-8969-ca99e9240439 None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:51.051681 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b362b9-3dc9-4b6d-8969-ca99e9240439 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:51.051970 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b362b9-3dc9-4b6d-8969-ca99e9240439 None admin] PATH_INFO: `/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe/users/e0fac339988e453e8f6be2c7ba535863/roles/0f660bd0ff4c4beda97ed7c24851d2d4` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:51.054254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b362b9-3dc9-4b6d-8969-ca99e9240439 None admin] RBAC: Authorizing `identity:create_grant(project_id=e55f1cc9f9c9481698d6962c7af7fbfe, user_id=e0fac339988e453e8f6be2c7ba535863, role_id=0f660bd0ff4c4beda97ed7c24851d2d4)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:51.055869 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b362b9-3dc9-4b6d-8969-ca99e9240439 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:51.058445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:51.074736 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 71/148] 10.222.0.71 () {58 vars in 1439 bytes} [Fri Aug 26 08:36:50 2022] PUT /identity/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe/users/e0fac339988e453e8f6be2c7ba535863/roles/0f660bd0ff4c4beda97ed7c24851d2d4 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:36:51.093305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fea10-a5af-4470-98eb-5ecb44543b7c None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:51.093714 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fea10-a5af-4470-98eb-5ecb44543b7c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:51.094041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fea10-a5af-4470-98eb-5ecb44543b7c None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:51.095584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 78/149] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:51.102496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b6cd5-04b1-462d-8e8a-ea9a1fece9d2 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:51.102808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b6cd5-04b1-462d-8e8a-ea9a1fece9d2 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:51.103077 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b6cd5-04b1-462d-8e8a-ea9a1fece9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:51.132124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-8f5b6cd5-04b1-462d-8e8a-ea9a1fece9d2 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:51.133343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8f5b6cd5-04b1-462d-8e8a-ea9a1fece9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:51.224511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 72/150] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:36:51.574633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fea10-a5af-4470-98eb-5ecb44543b7c None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:51.701215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fea10-a5af-4470-98eb-5ecb44543b7c None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:51.726310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:51.740708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7a851-360f-4168-ac69-3d89f1e646bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:51.741461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7a851-360f-4168-ac69-3d89f1e646bb None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:51.741625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7a851-360f-4168-ac69-3d89f1e646bb None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:51.741868 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7a851-360f-4168-ac69-3d89f1e646bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:51.755373 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7a851-360f-4168-ac69-3d89f1e646bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:36:51.760488 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-c0d7a851-360f-4168-ac69-3d89f1e646bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:36:51.760488 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:36:51.760488 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:36:51.760488 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:36:51.760488 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:36:51.760488 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:36:51.760488 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:36:51.760488 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:36:51.760488 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:36:51.760488 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:36:51.760488 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:36:51.760488 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:36:51.760488 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:36:51.760488 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:36:51.760488 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:36:51.760488 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:36:51.762430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:36:51.762430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:36:51.762430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:36:51.762430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:36:51.762430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:36:51.762430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:36:51.762430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:36:51.762430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:36:51.762430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:36:51.762430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:36:51.762430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:36:51.762430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:36:51.762430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:36:51.762430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:36:51.762430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:36:51.762430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:36:51.764262 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:36:51.764262 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:36:51.764262 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:36:51.764262 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:36:51.764262 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:36:51.764262 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:36:51.764262 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:36:51.764262 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:36:51.764262 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:36:51.764262 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:36:51.764262 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:36:51.764262 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:36:51.764262 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 79/151] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:36:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:36:51.772098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b6cd5-04b1-462d-8e8a-ea9a1fece9d2 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:51.904083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b6cd5-04b1-462d-8e8a-ea9a1fece9d2 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:51.931748 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:51.944422 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb4239-a3f1-48a2-a966-6ba314238fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:51.945102 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb4239-a3f1-48a2-a966-6ba314238fdb None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:51.945377 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb4239-a3f1-48a2-a966-6ba314238fdb None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:51.945643 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb4239-a3f1-48a2-a966-6ba314238fdb None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:51.946355 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cb4239-a3f1-48a2-a966-6ba314238fdb None admin] RBAC: Adding query filter params (name=admin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:51.946653 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cb4239-a3f1-48a2-a966-6ba314238fdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:51.948096 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cb4239-a3f1-48a2-a966-6ba314238fdb None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:51.957986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 73/152] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:36:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:51.963162 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7a851-360f-4168-ac69-3d89f1e646bb None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:52.089855 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7a851-360f-4168-ac69-3d89f1e646bb None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:52.128842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550a358-6fa3-45ec-9a2a-29192d9e2308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:52.129723 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550a358-6fa3-45ec-9a2a-29192d9e2308 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:52.130160 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550a358-6fa3-45ec-9a2a-29192d9e2308 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:52.130566 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550a358-6fa3-45ec-9a2a-29192d9e2308 None admin] PATH_INFO: `/v3/users/668b82af2c1345cca572ef92fff86ff2` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:52.131879 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1550a358-6fa3-45ec-9a2a-29192d9e2308 None admin] RBAC: Authorizing `identity:get_user(user_id=668b82af2c1345cca572ef92fff86ff2)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:52.133469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1550a358-6fa3-45ec-9a2a-29192d9e2308 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:52.135266 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 80/153] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:36:51 2022] GET /identity/v3/users/668b82af2c1345cca572ef92fff86ff2 => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:52.141021 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb4239-a3f1-48a2-a966-6ba314238fdb None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:52.294812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb4239-a3f1-48a2-a966-6ba314238fdb None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:52.337895 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530a0ce-0870-4f5c-8bdc-eb189039dd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:52.338514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530a0ce-0870-4f5c-8bdc-eb189039dd8c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:52.338794 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530a0ce-0870-4f5c-8bdc-eb189039dd8c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:52.339310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530a0ce-0870-4f5c-8bdc-eb189039dd8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:52.341057 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfdc6b-a4cf-4ed5-b192-20599199e76b None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:52.341575 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfdc6b-a4cf-4ed5-b192-20599199e76b None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:52.342005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfdc6b-a4cf-4ed5-b192-20599199e76b None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:52.347382 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 81/154] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:52.354171 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837415c-7e66-48c8-aa7c-587f0ecc93e9 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:52.354716 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837415c-7e66-48c8-aa7c-587f0ecc93e9 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:52.355168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837415c-7e66-48c8-aa7c-587f0ecc93e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:52.386620 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-6837415c-7e66-48c8-aa7c-587f0ecc93e9 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:52.388225 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-6837415c-7e66-48c8-aa7c-587f0ecc93e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:52.470032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:52.470503 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:52.505826 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530a0ce-0870-4f5c-8bdc-eb189039dd8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:52.507329 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530a0ce-0870-4f5c-8bdc-eb189039dd8c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:52.510193 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 82/155] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:36:52.649452 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 74/156] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:36:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:36:52.655349 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837415c-7e66-48c8-aa7c-587f0ecc93e9 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:52.668750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f56632-0ca5-4329-88a5-a3421ae1773e None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:52.669348 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f56632-0ca5-4329-88a5-a3421ae1773e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:52.671322 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f56632-0ca5-4329-88a5-a3421ae1773e None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:52.672947 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 75/157] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:52.678410 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdd8a5-634a-4beb-b47a-48f1824cec29 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:52.679023 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdd8a5-634a-4beb-b47a-48f1824cec29 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:52.679671 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdd8a5-634a-4beb-b47a-48f1824cec29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:52.717335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-1fcdd8a5-634a-4beb-b47a-48f1824cec29 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:52.720709 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1fcdd8a5-634a-4beb-b47a-48f1824cec29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:52.852324 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 76/158] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:36:52.868033 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6837415c-7e66-48c8-aa7c-587f0ecc93e9 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:52.917354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b4fabb-9b93-47e3-a7ff-f0a6abf1cec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:52.917939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4fabb-9b93-47e3-a7ff-f0a6abf1cec4 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:52.918213 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4fabb-9b93-47e3-a7ff-f0a6abf1cec4 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:52.918482 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4fabb-9b93-47e3-a7ff-f0a6abf1cec4 None admin] PATH_INFO: `/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:52.919909 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2a727-7b1f-42c6-9a4a-92b8ea9c9cd1 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:52.920720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2a727-7b1f-42c6-9a4a-92b8ea9c9cd1 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:52.921173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2a727-7b1f-42c6-9a4a-92b8ea9c9cd1 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:52.921785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b4fabb-9b93-47e3-a7ff-f0a6abf1cec4 None admin] RBAC: Authorizing `identity:get_project(project_id=e55f1cc9f9c9481698d6962c7af7fbfe)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:52.923254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 77/159] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:52.923431 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b4fabb-9b93-47e3-a7ff-f0a6abf1cec4 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:52.925601 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 83/160] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:36:52 2022] GET /identity/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:52.929447 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9536ff-9bdf-444d-b5ad-3089bb487482 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:52.930011 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9536ff-9bdf-444d-b5ad-3089bb487482 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:52.930466 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9536ff-9bdf-444d-b5ad-3089bb487482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:52.930925 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b4fabb-9b93-47e3-a7ff-f0a6abf1cec4 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:52.969736 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ab9536ff-9bdf-444d-b5ad-3089bb487482 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:52.971740 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ab9536ff-9bdf-444d-b5ad-3089bb487482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:52.993214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:53.028997 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:53.112226 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 78/161] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:36:53.138664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b4fabb-9b93-47e3-a7ff-f0a6abf1cec4 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:53.164862 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9536ff-9bdf-444d-b5ad-3089bb487482 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:53.214058 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6f7e5-3519-4e2c-9bdc-7283485990f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:53.214617 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6f7e5-3519-4e2c-9bdc-7283485990f4 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:53.214909 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6f7e5-3519-4e2c-9bdc-7283485990f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:53.215191 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6f7e5-3519-4e2c-9bdc-7283485990f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:53.216155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f6f7e5-3519-4e2c-9bdc-7283485990f4 None admin] RBAC: Adding query filter params (role.id=18c3c8a821b647f7982a955880bd0a9a, scope.project.id=e55f1cc9f9c9481698d6962c7af7fbfe, user.id=668b82af2c1345cca572ef92fff86ff2) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:53.216466 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f6f7e5-3519-4e2c-9bdc-7283485990f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:53.217988 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f6f7e5-3519-4e2c-9bdc-7283485990f4 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:53.227413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 84/162] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:36:52 2022] GET /identity/v3/role_assignments?user.id=668b82af2c1345cca572ef92fff86ff2&scope.project.id=e55f1cc9f9c9481698d6962c7af7fbfe&role.id=18c3c8a821b647f7982a955880bd0a9a => generated 608 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:36:53.345449 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9536ff-9bdf-444d-b5ad-3089bb487482 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:53.440105 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c155826-f07c-4757-aa5d-eecd17d79393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:53.441109 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c155826-f07c-4757-aa5d-eecd17d79393 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:53.441529 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c155826-f07c-4757-aa5d-eecd17d79393 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:53.441980 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c155826-f07c-4757-aa5d-eecd17d79393 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:53.468056 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c155826-f07c-4757-aa5d-eecd17d79393 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:36:53.475297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-5c155826-f07c-4757-aa5d-eecd17d79393 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 08:36:53.475297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:36:53.475297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:36:53.475297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:36:53.475297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:36:53.475297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:36:53.475297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:36:53.475297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:36:53.475297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:36:53.475297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:36:53.475297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:36:53.475297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:36:53.475297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:36:53.475297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:36:53.475297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:36:53.475297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:36:53.476966 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:36:53.476966 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:36:53.476966 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:36:53.476966 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:36:53.476966 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:36:53.476966 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:36:53.476966 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:36:53.476966 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:36:53.476966 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:36:53.476966 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:36:53.476966 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:36:53.476966 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:36:53.476966 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:36:53.476966 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:36:53.476966 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:36:53.476966 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:36:53.478431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:36:53.478431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:36:53.478431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:36:53.478431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:36:53.478431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:36:53.478431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:36:53.478431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:36:53.478431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:36:53.478431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:36:53.478431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:36:53.478431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 08:36:53.478431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:36:53.478431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 79/163] 10.222.0.71 () {56 vars in 1114 bytes} [Fri Aug 26 08:36:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:36:53.488174 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6f7e5-3519-4e2c-9bdc-7283485990f4 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:53.631174 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c155826-f07c-4757-aa5d-eecd17d79393 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:53.640920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:53.658384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6f7e5-3519-4e2c-9bdc-7283485990f4 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:53.692195 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:53.772177 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f95ca-9aa6-42ec-b4ea-6899c89a5a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:53.773159 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f95ca-9aa6-42ec-b4ea-6899c89a5a20 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:53.773626 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f95ca-9aa6-42ec-b4ea-6899c89a5a20 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:53.774100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f95ca-9aa6-42ec-b4ea-6899c89a5a20 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:53.775070 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f95ca-9aa6-42ec-b4ea-6899c89a5a20 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:53.775643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f95ca-9aa6-42ec-b4ea-6899c89a5a20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:53.777243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f95ca-9aa6-42ec-b4ea-6899c89a5a20 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:53.794344 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 85/164] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:36:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:53.800583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f95ca-9aa6-42ec-b4ea-6899c89a5a20 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:53.819274 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c155826-f07c-4757-aa5d-eecd17d79393 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:53.872588 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499121ac-9011-4c46-91eb-c3f71ff4ab3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:53.873238 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499121ac-9011-4c46-91eb-c3f71ff4ab3d None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:53.873517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499121ac-9011-4c46-91eb-c3f71ff4ab3d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:53.873789 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499121ac-9011-4c46-91eb-c3f71ff4ab3d None admin] PATH_INFO: `/v3/roles/member` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:53.902156 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499121ac-9011-4c46-91eb-c3f71ff4ab3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:36:53.907025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-499121ac-9011-4c46-91eb-c3f71ff4ab3d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:36:53.907025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:36:53.907025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:36:53.907025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:36:53.907025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:36:53.907025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:36:53.907025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:36:53.907025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:36:53.907025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:36:53.907025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:36:53.907025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:36:53.907025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:36:53.907025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:36:53.907025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:36:53.907025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:36:53.907025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:36:53.908664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:36:53.908664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:36:53.908664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:36:53.908664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:36:53.908664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:36:53.908664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:36:53.908664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:36:53.908664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:36:53.908664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:36:53.908664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:36:53.908664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:36:53.908664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:36:53.908664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:36:53.908664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:36:53.908664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:36:53.908664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:36:53.909512 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:36:53.909512 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:36:53.909512 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:36:53.909512 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:36:53.909512 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:36:53.909512 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:36:53.909512 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:36:53.909512 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:36:53.909512 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:36:53.909512 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:36:53.909512 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:36:53.909512 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:36:53.915431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 80/165] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:36:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:36:53.917395 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499121ac-9011-4c46-91eb-c3f71ff4ab3d None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:54.001381 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f95ca-9aa6-42ec-b4ea-6899c89a5a20 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:54.043524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:54.071736 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9ed63-e26c-4595-bacc-cb6a34bda5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:54.072418 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9ed63-e26c-4595-bacc-cb6a34bda5b4 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:54.072708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9ed63-e26c-4595-bacc-cb6a34bda5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:54.072994 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9ed63-e26c-4595-bacc-cb6a34bda5b4 None admin] PATH_INFO: `/v3/users/748c27d9ec1c4f29912ec71cb6f8863c` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:54.074786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9ed63-e26c-4595-bacc-cb6a34bda5b4 None admin] RBAC: Authorizing `identity:get_user(user_id=748c27d9ec1c4f29912ec71cb6f8863c)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:54.077271 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9ed63-e26c-4595-bacc-cb6a34bda5b4 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:54.080328 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 86/166] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:36:53 2022] GET /identity/v3/users/748c27d9ec1c4f29912ec71cb6f8863c => generated 275 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:54.081197 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9ed63-e26c-4595-bacc-cb6a34bda5b4 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:54.137296 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499121ac-9011-4c46-91eb-c3f71ff4ab3d None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:54.167277 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:54.263755 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b20901-06a4-4f2e-985d-9936855567c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:54.264437 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b20901-06a4-4f2e-985d-9936855567c8 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:54.264770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b20901-06a4-4f2e-985d-9936855567c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:54.265044 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b20901-06a4-4f2e-985d-9936855567c8 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:54.265899 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b20901-06a4-4f2e-985d-9936855567c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:54.266213 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b20901-06a4-4f2e-985d-9936855567c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:54.267799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b20901-06a4-4f2e-985d-9936855567c8 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:54.279967 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 81/167] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:36:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:54.284101 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b20901-06a4-4f2e-985d-9936855567c8 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:54.321544 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9ed63-e26c-4595-bacc-cb6a34bda5b4 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:54.427783 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c282d2-e295-4b64-a88a-befc9b7474af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:54.428571 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c282d2-e295-4b64-a88a-befc9b7474af None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:54.428850 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c282d2-e295-4b64-a88a-befc9b7474af None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:54.429128 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c282d2-e295-4b64-a88a-befc9b7474af None admin] PATH_INFO: `/v3/roles/member` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:54.449793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c282d2-e295-4b64-a88a-befc9b7474af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:36:54.460097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-87c282d2-e295-4b64-a88a-befc9b7474af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:36:54.460097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:36:54.460097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:36:54.460097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:36:54.460097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:36:54.460097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:36:54.460097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:36:54.460097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:36:54.460097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:36:54.460097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:36:54.460097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:36:54.460097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:36:54.460097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:36:54.460097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:36:54.460097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:36:54.460097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:36:54.461700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:36:54.461700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:36:54.461700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:36:54.461700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:36:54.461700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:36:54.461700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:36:54.461700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:36:54.461700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:36:54.461700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:36:54.461700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:36:54.461700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:36:54.461700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:36:54.461700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:36:54.461700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:36:54.461700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:36:54.461700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:36:54.463060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:36:54.463060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:36:54.463060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:36:54.463060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:36:54.463060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:36:54.463060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:36:54.463060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:36:54.463060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:36:54.463060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:36:54.463060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:36:54.463060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:36:54.463060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:36:54.463060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 87/168] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:36:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:36:54.464885 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c282d2-e295-4b64-a88a-befc9b7474af None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:54.618150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b20901-06a4-4f2e-985d-9936855567c8 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:54.711964 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:54.717268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dfbd27-dcd7-4564-bf99-06ce7831a351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:54.718159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dfbd27-dcd7-4564-bf99-06ce7831a351 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:54.718459 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dfbd27-dcd7-4564-bf99-06ce7831a351 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:54.718780 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dfbd27-dcd7-4564-bf99-06ce7831a351 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:54.733341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c282d2-e295-4b64-a88a-befc9b7474af None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:54.737150 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:54.824714 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d89ea-ec1b-4393-9c11-3a1d7b746157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:54.825388 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d89ea-ec1b-4393-9c11-3a1d7b746157 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:54.825756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d89ea-ec1b-4393-9c11-3a1d7b746157 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:54.826024 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d89ea-ec1b-4393-9c11-3a1d7b746157 None admin] PATH_INFO: `/v3/users/748c27d9ec1c4f29912ec71cb6f8863c` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:54.830327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d89ea-ec1b-4393-9c11-3a1d7b746157 None admin] RBAC: Authorizing `identity:get_user(user_id=748c27d9ec1c4f29912ec71cb6f8863c)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:54.831923 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d89ea-ec1b-4393-9c11-3a1d7b746157 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:54.833867 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 88/169] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:36:54 2022] GET /identity/v3/users/748c27d9ec1c4f29912ec71cb6f8863c => generated 275 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:54.839700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d89ea-ec1b-4393-9c11-3a1d7b746157 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:54.928528 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dfbd27-dcd7-4564-bf99-06ce7831a351 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:54.930769 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dfbd27-dcd7-4564-bf99-06ce7831a351 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:55.099795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d89ea-ec1b-4393-9c11-3a1d7b746157 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:55.170157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6278471-0e05-4084-93da-93c873187d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:55.170978 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6278471-0e05-4084-93da-93c873187d57 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:55.171486 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6278471-0e05-4084-93da-93c873187d57 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:55.171780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6278471-0e05-4084-93da-93c873187d57 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:55.172738 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6278471-0e05-4084-93da-93c873187d57 None admin] RBAC: Adding query filter params (name=member) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:55.173235 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6278471-0e05-4084-93da-93c873187d57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:55.175633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6278471-0e05-4084-93da-93c873187d57 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:55.188340 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 89/170] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:36:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:55.190020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6278471-0e05-4084-93da-93c873187d57 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:55.235289 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:55.241965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 82/171] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:36:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 962 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:36:55.244342 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd1a7e-fe20-44e8-a3b6-d98c5b8b9d7c None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:55.244822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd1a7e-fe20-44e8-a3b6-d98c5b8b9d7c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:55.245296 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd1a7e-fe20-44e8-a3b6-d98c5b8b9d7c None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:55.256132 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 83/172] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:55 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:55.257600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd1a7e-fe20-44e8-a3b6-d98c5b8b9d7c None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:55.319409 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:55.349652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6278471-0e05-4084-93da-93c873187d57 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:55.393814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd6893-c651-4dcc-aff9-c808d37e6a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:55.394608 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd6893-c651-4dcc-aff9-c808d37e6a20 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:55.394793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd6893-c651-4dcc-aff9-c808d37e6a20 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:55.395087 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd6893-c651-4dcc-aff9-c808d37e6a20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:55.464355 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd1a7e-fe20-44e8-a3b6-d98c5b8b9d7c None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:55.501337 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd6893-c651-4dcc-aff9-c808d37e6a20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:55.504101 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd6893-c651-4dcc-aff9-c808d37e6a20 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:55.534654 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd301034-729c-4ec6-a2d4-78a0eab83a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:55.535745 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd301034-729c-4ec6-a2d4-78a0eab83a34 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:55.536231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd301034-729c-4ec6-a2d4-78a0eab83a34 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:55.536711 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd301034-729c-4ec6-a2d4-78a0eab83a34 None admin] PATH_INFO: `/v3/users/748c27d9ec1c4f29912ec71cb6f8863c` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:55.538947 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd301034-729c-4ec6-a2d4-78a0eab83a34 None admin] RBAC: Authorizing `identity:get_user(user_id=748c27d9ec1c4f29912ec71cb6f8863c)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:55.541757 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd301034-729c-4ec6-a2d4-78a0eab83a34 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:55.544778 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 84/173] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:36:55 2022] GET /identity/v3/users/748c27d9ec1c4f29912ec71cb6f8863c => generated 275 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:55.545800 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd301034-729c-4ec6-a2d4-78a0eab83a34 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:55.556985 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:55.678972 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 90/174] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:36:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:36:55.681960 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5335e40-3576-4b07-a013-b6adbdb942a7 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:55.682455 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5335e40-3576-4b07-a013-b6adbdb942a7 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:55.682976 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5335e40-3576-4b07-a013-b6adbdb942a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:55.713085 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:55.715268 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e5335e40-3576-4b07-a013-b6adbdb942a7 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:55.716935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e5335e40-3576-4b07-a013-b6adbdb942a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:55.728275 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd301034-729c-4ec6-a2d4-78a0eab83a34 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:55.801937 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d72ea2b-a0fe-42b4-a197-5adcbf9c69b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:55.802967 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72ea2b-a0fe-42b4-a197-5adcbf9c69b0 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:55.803608 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72ea2b-a0fe-42b4-a197-5adcbf9c69b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:55.803765 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72ea2b-a0fe-42b4-a197-5adcbf9c69b0 None admin] PATH_INFO: `/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:55.805025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d72ea2b-a0fe-42b4-a197-5adcbf9c69b0 None admin] RBAC: Authorizing `identity:get_project(project_id=e55f1cc9f9c9481698d6962c7af7fbfe)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:55.807147 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d72ea2b-a0fe-42b4-a197-5adcbf9c69b0 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:55.810113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 85/175] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:36:55 2022] GET /identity/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:55.812515 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f161c-c9ea-48b4-ae57-730a4139374e None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:55.812995 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f161c-c9ea-48b4-ae57-730a4139374e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:55.813442 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f161c-c9ea-48b4-ae57-730a4139374e None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:55.818384 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 86/176] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:55.818384 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d543ad1-bf79-470e-8529-abb449544830 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:55.818753 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d543ad1-bf79-470e-8529-abb449544830 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:55.819246 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d543ad1-bf79-470e-8529-abb449544830 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:55.821693 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 87/177] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:55.827547 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d543ad1-bf79-470e-8529-abb449544830 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:55.843049 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 91/178] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:36:55.844078 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5335e40-3576-4b07-a013-b6adbdb942a7 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:55.963552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:56.001380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d543ad1-bf79-470e-8529-abb449544830 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:56.035958 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5335e40-3576-4b07-a013-b6adbdb942a7 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:56.065908 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ce596-9a2a-4836-9e43-b6ace5d43bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:56.066772 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ce596-9a2a-4836-9e43-b6ace5d43bec None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:56.067239 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ce596-9a2a-4836-9e43-b6ace5d43bec None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:56.067630 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ce596-9a2a-4836-9e43-b6ace5d43bec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:56.101748 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae635fc7-85c1-4020-be5c-5eb1a1187fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:56.103388 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae635fc7-85c1-4020-be5c-5eb1a1187fef None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:56.103388 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae635fc7-85c1-4020-be5c-5eb1a1187fef None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:56.103690 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae635fc7-85c1-4020-be5c-5eb1a1187fef None admin] PATH_INFO: `/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:56.105737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae635fc7-85c1-4020-be5c-5eb1a1187fef None admin] RBAC: Authorizing `identity:get_project(project_id=e55f1cc9f9c9481698d6962c7af7fbfe)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:56.108090 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae635fc7-85c1-4020-be5c-5eb1a1187fef None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:56.109869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 92/179] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:36:55 2022] GET /identity/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:56.110778 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae635fc7-85c1-4020-be5c-5eb1a1187fef None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:56.125425 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:56.203113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27ce596-9a2a-4836-9e43-b6ace5d43bec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:56.205683 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27ce596-9a2a-4836-9e43-b6ace5d43bec None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:56.277918 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae635fc7-85c1-4020-be5c-5eb1a1187fef None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:56.327491 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:56.334867 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c083d4-5175-4f58-8b68-6d21b4d49e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:56.335625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c083d4-5175-4f58-8b68-6d21b4d49e17 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:56.335909 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c083d4-5175-4f58-8b68-6d21b4d49e17 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:56.336171 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c083d4-5175-4f58-8b68-6d21b4d49e17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:56.336996 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c083d4-5175-4f58-8b68-6d21b4d49e17 None admin] RBAC: Adding query filter params (role.id=6195dc21ae4749daa31797d2c42e050e, scope.project.id=e55f1cc9f9c9481698d6962c7af7fbfe, user.id=748c27d9ec1c4f29912ec71cb6f8863c) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:56.337282 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c083d4-5175-4f58-8b68-6d21b4d49e17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:56.338857 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c083d4-5175-4f58-8b68-6d21b4d49e17 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:56.344692 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 88/180] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:36:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:36:56.352537 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 93/181] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:36:56 2022] GET /identity/v3/role_assignments?user.id=748c27d9ec1c4f29912ec71cb6f8863c&scope.project.id=e55f1cc9f9c9481698d6962c7af7fbfe&role.id=6195dc21ae4749daa31797d2c42e050e => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:56.352757 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af13e6-23c5-4f2e-90d9-6ef93568bf67 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:56.352850 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7694e-8670-4771-b245-5093c00bb64a None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:56.352850 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7694e-8670-4771-b245-5093c00bb64a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:56.352850 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7694e-8670-4771-b245-5093c00bb64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:56.353120 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af13e6-23c5-4f2e-90d9-6ef93568bf67 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:56.353120 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af13e6-23c5-4f2e-90d9-6ef93568bf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:56.374664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-b3e7694e-8670-4771-b245-5093c00bb64a None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:56.376062 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b3e7694e-8670-4771-b245-5093c00bb64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:56.386373 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-a7af13e6-23c5-4f2e-90d9-6ef93568bf67 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:56.388318 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a7af13e6-23c5-4f2e-90d9-6ef93568bf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:56.486811 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:56.488520 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 94/182] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:36:56.490134 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633297a-62d6-4d95-afa9-1638f4e9ab3e None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:56.490470 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633297a-62d6-4d95-afa9-1638f4e9ab3e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:56.490750 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633297a-62d6-4d95-afa9-1638f4e9ab3e None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:56.492455 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 95/183] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:36:56.493041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633297a-62d6-4d95-afa9-1638f4e9ab3e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:56.516874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 89/184] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:36:56.518865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af13e6-23c5-4f2e-90d9-6ef93568bf67 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:56.684623 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633297a-62d6-4d95-afa9-1638f4e9ab3e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:56.706472 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af13e6-23c5-4f2e-90d9-6ef93568bf67 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:56.726939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:56.744588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2c032-3d17-475c-aa1a-7129b3eef71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:56.745989 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2c032-3d17-475c-aa1a-7129b3eef71c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:56.746285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2c032-3d17-475c-aa1a-7129b3eef71c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:56.746557 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2c032-3d17-475c-aa1a-7129b3eef71c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:56.747351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc2c032-3d17-475c-aa1a-7129b3eef71c None admin] RBAC: Adding query filter params (role.id=b7a48a25b0cd479d85fe5395d64c458e, scope.project.id=e55f1cc9f9c9481698d6962c7af7fbfe, user.id=748c27d9ec1c4f29912ec71cb6f8863c) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:56.747712 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc2c032-3d17-475c-aa1a-7129b3eef71c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:56.749482 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc2c032-3d17-475c-aa1a-7129b3eef71c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:56.762917 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 96/185] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:36:56 2022] GET /identity/v3/role_assignments?user.id=748c27d9ec1c4f29912ec71cb6f8863c&scope.project.id=e55f1cc9f9c9481698d6962c7af7fbfe&role.id=b7a48a25b0cd479d85fe5395d64c458e => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:56.762917 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2c032-3d17-475c-aa1a-7129b3eef71c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:56.780991 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0256bd5b-9914-4d14-a84d-c1868981a989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:56.782093 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256bd5b-9914-4d14-a84d-c1868981a989 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:56.782594 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256bd5b-9914-4d14-a84d-c1868981a989 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:56.783171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0256bd5b-9914-4d14-a84d-c1868981a989 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:56.804713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0256bd5b-9914-4d14-a84d-c1868981a989 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:36:56.810390 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-0256bd5b-9914-4d14-a84d-c1868981a989 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:36:56.810390 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:36:56.810390 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:36:56.810390 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:36:56.810390 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:36:56.810390 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:36:56.810390 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:36:56.810390 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:36:56.810390 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:36:56.810390 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:36:56.810390 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:36:56.810390 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:36:56.810390 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:36:56.810390 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:36:56.810390 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:36:56.810390 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:36:56.814510 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:36:56.814510 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:36:56.814510 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:36:56.814510 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:36:56.814510 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:36:56.814510 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:36:56.814510 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:36:56.814510 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:36:56.814510 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:36:56.814510 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:36:56.814510 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:36:56.814510 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:36:56.814510 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:36:56.814510 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:36:56.814510 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:36:56.814510 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:36:56.815576 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:36:56.815576 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:36:56.815576 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:36:56.815576 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:36:56.815576 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:36:56.815576 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:36:56.815576 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:36:56.815576 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:36:56.815576 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:36:56.815576 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:36:56.815576 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:36:56.815576 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:36:56.818864 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 90/186] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:36:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:36:56.820973 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd50ed5-9cfd-452e-a9ac-2e1d70b3f217 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:56.821406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd50ed5-9cfd-452e-a9ac-2e1d70b3f217 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:56.821816 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd50ed5-9cfd-452e-a9ac-2e1d70b3f217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:56.864364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-efd50ed5-9cfd-452e-a9ac-2e1d70b3f217 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:36:56.866364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-efd50ed5-9cfd-452e-a9ac-2e1d70b3f217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:36:56.918544 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2c032-3d17-475c-aa1a-7129b3eef71c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:56.976337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:56.991214 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45244cc-ac92-4795-bf71-8fbf11c444b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:56.992137 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45244cc-ac92-4795-bf71-8fbf11c444b6 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:56.992418 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45244cc-ac92-4795-bf71-8fbf11c444b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:56.992694 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45244cc-ac92-4795-bf71-8fbf11c444b6 None admin] PATH_INFO: `/v3/projects/ead3f51f90b34ba6852874282e70d280` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:56.994228 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45244cc-ac92-4795-bf71-8fbf11c444b6 None admin] RBAC: Authorizing `identity:get_project(project_id=ead3f51f90b34ba6852874282e70d280)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:56.995772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45244cc-ac92-4795-bf71-8fbf11c444b6 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:56.997798 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 97/187] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:36:56 2022] GET /identity/v3/projects/ead3f51f90b34ba6852874282e70d280 => generated 312 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:57.004080 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45244cc-ac92-4795-bf71-8fbf11c444b6 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:57.012373 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 91/188] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:36:57.013630 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd50ed5-9cfd-452e-a9ac-2e1d70b3f217 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:57.106082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:57.197390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45244cc-ac92-4795-bf71-8fbf11c444b6 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:57.198968 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd50ed5-9cfd-452e-a9ac-2e1d70b3f217 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:57.254045 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfef361-11a8-4953-bdc3-2be25ae5d004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:57.254669 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfef361-11a8-4953-bdc3-2be25ae5d004 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:57.254996 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfef361-11a8-4953-bdc3-2be25ae5d004 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:57.255277 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfef361-11a8-4953-bdc3-2be25ae5d004 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:57.256660 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1fe7b1-c79d-4abc-8ba4-69ffc6985291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:57.257843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fe7b1-c79d-4abc-8ba4-69ffc6985291 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:57.258694 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fe7b1-c79d-4abc-8ba4-69ffc6985291 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:57.259175 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1fe7b1-c79d-4abc-8ba4-69ffc6985291 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:57.260624 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1fe7b1-c79d-4abc-8ba4-69ffc6985291 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:57.261506 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1fe7b1-c79d-4abc-8ba4-69ffc6985291 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:57.264519 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1fe7b1-c79d-4abc-8ba4-69ffc6985291 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:57.269317 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfef361-11a8-4953-bdc3-2be25ae5d004 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:36:57.275028 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-4cfef361-11a8-4953-bdc3-2be25ae5d004 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 08:36:57.275028 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:36:57.275028 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:36:57.275028 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:36:57.275028 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:36:57.275028 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:36:57.275028 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:36:57.275028 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:36:57.275028 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:36:57.275028 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:36:57.275028 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:36:57.275028 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:36:57.275028 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:36:57.275028 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:36:57.275028 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:36:57.275028 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:36:57.276657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:36:57.276657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:36:57.276657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:36:57.276657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:36:57.276657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:36:57.276657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:36:57.276657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:36:57.276657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:36:57.276657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:36:57.276657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:36:57.276657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:36:57.276657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:36:57.276657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:36:57.276657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:36:57.276657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:36:57.276657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:36:57.278254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:36:57.278254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:36:57.278254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:36:57.278254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:36:57.278254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:36:57.278254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:36:57.278254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:36:57.278254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:36:57.278254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:36:57.278254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:36:57.278254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 08:36:57.278254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:36:57.278254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 92/189] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:36:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:36:57.278254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfef361-11a8-4953-bdc3-2be25ae5d004 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:57.290473 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 98/190] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:36:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:57.291795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1fe7b1-c79d-4abc-8ba4-69ffc6985291 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:57.382125 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:57.471631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1fe7b1-c79d-4abc-8ba4-69ffc6985291 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:57.483366 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfef361-11a8-4953-bdc3-2be25ae5d004 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:57.509042 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:57.545564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4ce9e-b639-44b9-8611-d8a42260c603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:57.546550 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4ce9e-b639-44b9-8611-d8a42260c603 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:57.546996 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4ce9e-b639-44b9-8611-d8a42260c603 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:57.547404 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4ce9e-b639-44b9-8611-d8a42260c603 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:57.548402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e4ce9e-b639-44b9-8611-d8a42260c603 None admin] RBAC: Adding query filter params (role.id=b7a48a25b0cd479d85fe5395d64c458e, scope.project.id=ead3f51f90b34ba6852874282e70d280, user.id=748c27d9ec1c4f29912ec71cb6f8863c) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:57.548815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e4ce9e-b639-44b9-8611-d8a42260c603 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:57.548934 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5525d4d-8fe5-48cc-a3e2-f0ba78ee3917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:57.549899 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5525d4d-8fe5-48cc-a3e2-f0ba78ee3917 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:57.550251 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5525d4d-8fe5-48cc-a3e2-f0ba78ee3917 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:57.550511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5525d4d-8fe5-48cc-a3e2-f0ba78ee3917 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:57.551108 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e4ce9e-b639-44b9-8611-d8a42260c603 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:57.552958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5525d4d-8fe5-48cc-a3e2-f0ba78ee3917 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:57.555276 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5525d4d-8fe5-48cc-a3e2-f0ba78ee3917 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:57.557538 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 93/191] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:36:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:57.558573 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5525d4d-8fe5-48cc-a3e2-f0ba78ee3917 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:57.565025 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 99/192] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:36:57 2022] GET /identity/v3/role_assignments?user.id=748c27d9ec1c4f29912ec71cb6f8863c&scope.project.id=ead3f51f90b34ba6852874282e70d280&role.id=b7a48a25b0cd479d85fe5395d64c458e => generated 608 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:57.568270 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4ce9e-b639-44b9-8611-d8a42260c603 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:57.761688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5525d4d-8fe5-48cc-a3e2-f0ba78ee3917 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:57.790071 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:57.790764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4ce9e-b639-44b9-8611-d8a42260c603 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:57.826707 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcbb43-23f1-413b-9acc-bed7bc836987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:57.828604 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcbb43-23f1-413b-9acc-bed7bc836987 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:57.828604 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcbb43-23f1-413b-9acc-bed7bc836987 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:57.828604 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcbb43-23f1-413b-9acc-bed7bc836987 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:57.829204 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddcbb43-23f1-413b-9acc-bed7bc836987 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:57.829610 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddcbb43-23f1-413b-9acc-bed7bc836987 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:57.831864 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddcbb43-23f1-413b-9acc-bed7bc836987 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:57.846049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 94/193] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:36:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:57.851021 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcbb43-23f1-413b-9acc-bed7bc836987 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:57.853454 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ec5fb-fba8-4897-8002-81b6886f4bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:57.854716 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ec5fb-fba8-4897-8002-81b6886f4bb4 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:57.854716 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ec5fb-fba8-4897-8002-81b6886f4bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:57.855666 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ec5fb-fba8-4897-8002-81b6886f4bb4 None admin] PATH_INFO: `/v3/users/02a6d62d51854d3bb5bb2365bda8e8cb` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:57.872477 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831ec5fb-fba8-4897-8002-81b6886f4bb4 None admin] RBAC: Authorizing `identity:get_user(user_id=02a6d62d51854d3bb5bb2365bda8e8cb)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:57.874816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831ec5fb-fba8-4897-8002-81b6886f4bb4 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:57.878131 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 100/194] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:36:57 2022] GET /identity/v3/users/02a6d62d51854d3bb5bb2365bda8e8cb => generated 283 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:57.878131 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ec5fb-fba8-4897-8002-81b6886f4bb4 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:57.938781 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:57.994317 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcbb43-23f1-413b-9acc-bed7bc836987 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:58.016562 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ec5fb-fba8-4897-8002-81b6886f4bb4 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:58.042918 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ee6a0-65dd-4275-8a2b-5ad6e40aafc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:58.046278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ee6a0-65dd-4275-8a2b-5ad6e40aafc2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:58.046278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ee6a0-65dd-4275-8a2b-5ad6e40aafc2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:58.046278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ee6a0-65dd-4275-8a2b-5ad6e40aafc2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:58.058363 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ee6a0-65dd-4275-8a2b-5ad6e40aafc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:36:58.063944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-257ee6a0-65dd-4275-8a2b-5ad6e40aafc2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 08:36:58.063944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:36:58.063944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:36:58.063944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:36:58.063944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:36:58.063944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:36:58.063944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:36:58.063944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:36:58.063944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:36:58.063944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:36:58.063944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:36:58.063944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:36:58.063944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:36:58.063944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:36:58.063944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:36:58.063944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:36:58.067954 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:36:58.067954 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:36:58.067954 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:36:58.067954 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:36:58.067954 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:36:58.067954 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:36:58.067954 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:36:58.067954 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:36:58.067954 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:36:58.067954 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:36:58.067954 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:36:58.067954 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:36:58.067954 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:36:58.067954 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:36:58.067954 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:36:58.067954 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:36:58.069445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:36:58.069445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:36:58.069445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:36:58.069445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:36:58.069445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:36:58.069445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:36:58.069445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:36:58.069445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:36:58.069445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:36:58.069445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:36:58.069445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 08:36:58.069445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:36:58.069445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 95/195] 10.222.0.71 () {56 vars in 1114 bytes} [Fri Aug 26 08:36:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:36:58.069445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ee6a0-65dd-4275-8a2b-5ad6e40aafc2 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:58.073983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ed53c-bfe5-4548-b45c-56795a5fdf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:58.074627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ed53c-bfe5-4548-b45c-56795a5fdf89 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:58.074922 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ed53c-bfe5-4548-b45c-56795a5fdf89 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:58.075244 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ed53c-bfe5-4548-b45c-56795a5fdf89 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:58.075945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ed53c-bfe5-4548-b45c-56795a5fdf89 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:58.077532 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ed53c-bfe5-4548-b45c-56795a5fdf89 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:58.115506 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:58.115721 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 101/196] 10.222.0.71 () {60 vars in 1132 bytes} [Fri Aug 26 08:36:57 2022] POST /identity/v3/users => generated 297 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:36:58.116539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ed53c-bfe5-4548-b45c-56795a5fdf89 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:58.252397 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ee6a0-65dd-4275-8a2b-5ad6e40aafc2 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:58.309804 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:58.317574 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1c59e-606d-4535-b699-a55813a349ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:58.318559 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1c59e-606d-4535-b699-a55813a349ef None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:58.318933 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1c59e-606d-4535-b699-a55813a349ef None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:58.319142 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1c59e-606d-4535-b699-a55813a349ef None admin] PATH_INFO: `/v3/users/e0fac339988e453e8f6be2c7ba535863` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:58.320232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ed53c-bfe5-4548-b45c-56795a5fdf89 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:58.321282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd1c59e-606d-4535-b699-a55813a349ef None admin] RBAC: Authorizing `identity:get_user(user_id=e0fac339988e453e8f6be2c7ba535863)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:58.322833 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd1c59e-606d-4535-b699-a55813a349ef None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:58.325354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 96/197] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:36:58 2022] GET /identity/v3/users/e0fac339988e453e8f6be2c7ba535863 => generated 289 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:58.325713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1c59e-606d-4535-b699-a55813a349ef None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:58.399445 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb52073-ba2b-421c-b63f-6a5df2940af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:58.400514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb52073-ba2b-421c-b63f-6a5df2940af8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:58.401895 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb52073-ba2b-421c-b63f-6a5df2940af8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:58.401895 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb52073-ba2b-421c-b63f-6a5df2940af8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:58.508157 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1c59e-606d-4535-b699-a55813a349ef None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:58.511739 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:58.537321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb52073-ba2b-421c-b63f-6a5df2940af8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:58.539552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb52073-ba2b-421c-b63f-6a5df2940af8 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:58.591825 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a348e68-495e-4d3c-9a88-7c13f7f22889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:58.592390 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a348e68-495e-4d3c-9a88-7c13f7f22889 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:58.592725 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a348e68-495e-4d3c-9a88-7c13f7f22889 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:58.593037 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a348e68-495e-4d3c-9a88-7c13f7f22889 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:58.593822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a348e68-495e-4d3c-9a88-7c13f7f22889 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:58.594145 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a348e68-495e-4d3c-9a88-7c13f7f22889 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:58.596010 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a348e68-495e-4d3c-9a88-7c13f7f22889 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:58.612276 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 97/198] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:36:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:58.613310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a348e68-495e-4d3c-9a88-7c13f7f22889 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:58.712461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:58.724238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 102/199] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:36:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:36:58.725164 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb52073-ba2b-421c-b63f-6a5df2940af8 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:58.764955 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a348e68-495e-4d3c-9a88-7c13f7f22889 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:58.810616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c290cf51-e2bd-4928-9ac3-73b2cc2a1b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:58.811491 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290cf51-e2bd-4928-9ac3-73b2cc2a1b71 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:58.811773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290cf51-e2bd-4928-9ac3-73b2cc2a1b71 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:58.812048 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290cf51-e2bd-4928-9ac3-73b2cc2a1b71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:58.851654 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:58.883418 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb52073-ba2b-421c-b63f-6a5df2940af8 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:58.917497 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c290cf51-e2bd-4928-9ac3-73b2cc2a1b71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:58.919577 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c290cf51-e2bd-4928-9ac3-73b2cc2a1b71 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:58.939787 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a3dac-8b4a-420a-a67b-8b8852e9ebf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:58.941330 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a3dac-8b4a-420a-a67b-8b8852e9ebf0 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:58.942146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a3dac-8b4a-420a-a67b-8b8852e9ebf0 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:58.942801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a3dac-8b4a-420a-a67b-8b8852e9ebf0 None admin] PATH_INFO: `/v3/users/02a6d62d51854d3bb5bb2365bda8e8cb` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:58.944841 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a3dac-8b4a-420a-a67b-8b8852e9ebf0 None admin] RBAC: Authorizing `identity:get_user(user_id=02a6d62d51854d3bb5bb2365bda8e8cb)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:58.946820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a3dac-8b4a-420a-a67b-8b8852e9ebf0 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:58.949100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 103/200] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:36:58 2022] GET /identity/v3/users/02a6d62d51854d3bb5bb2365bda8e8cb => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:58.955239 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a3dac-8b4a-420a-a67b-8b8852e9ebf0 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:59.067509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:59.084693 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 98/201] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:36:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:36:59.086779 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c290cf51-e2bd-4928-9ac3-73b2cc2a1b71 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:59.141699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a3dac-8b4a-420a-a67b-8b8852e9ebf0 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:59.205524 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ede0c-6ede-4cbe-9b98-e2cb08e75a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:59.207270 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ede0c-6ede-4cbe-9b98-e2cb08e75a29 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:59.207744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ede0c-6ede-4cbe-9b98-e2cb08e75a29 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:59.208668 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ede0c-6ede-4cbe-9b98-e2cb08e75a29 None admin] PATH_INFO: `/v3/projects/214bc7d9ff5940989811937e08b70389` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:59.210557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:59.211859 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ede0c-6ede-4cbe-9b98-e2cb08e75a29 None admin] RBAC: Authorizing `identity:get_project(project_id=214bc7d9ff5940989811937e08b70389)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:59.214305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ede0c-6ede-4cbe-9b98-e2cb08e75a29 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:59.216992 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 104/202] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:36:58 2022] GET /identity/v3/projects/214bc7d9ff5940989811937e08b70389 => generated 302 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:59.218785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ede0c-6ede-4cbe-9b98-e2cb08e75a29 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:59.286748 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c290cf51-e2bd-4928-9ac3-73b2cc2a1b71 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:59.339677 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee0f05-07cf-455b-8ecf-a071010b4dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:59.340918 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee0f05-07cf-455b-8ecf-a071010b4dbd None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:59.341370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee0f05-07cf-455b-8ecf-a071010b4dbd None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:59.341845 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ee0f05-07cf-455b-8ecf-a071010b4dbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:59.405060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ede0c-6ede-4cbe-9b98-e2cb08e75a29 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:59.470569 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:59.471329 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ee0f05-07cf-455b-8ecf-a071010b4dbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:59.473718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ee0f05-07cf-455b-8ecf-a071010b4dbd None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:59.481945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e171a-9eae-4198-a9b0-44e302764c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:59.483287 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e171a-9eae-4198-a9b0-44e302764c76 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:59.483946 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e171a-9eae-4198-a9b0-44e302764c76 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:59.485439 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e171a-9eae-4198-a9b0-44e302764c76 None admin] PATH_INFO: `/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:59.487821 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e171a-9eae-4198-a9b0-44e302764c76 None admin] RBAC: Authorizing `identity:get_project(project_id=e55f1cc9f9c9481698d6962c7af7fbfe)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:59.490935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e171a-9eae-4198-a9b0-44e302764c76 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:59.493784 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 105/203] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:36:59 2022] GET /identity/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe => generated 298 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:59.495037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e171a-9eae-4198-a9b0-44e302764c76 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:59.584686 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:59.635619 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 99/204] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:36:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:36:59.636346 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee0f05-07cf-455b-8ecf-a071010b4dbd None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:59.641199 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e171a-9eae-4198-a9b0-44e302764c76 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:59.685254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a570398-18b8-46b3-8032-02156ff8636d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:59.686575 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a570398-18b8-46b3-8032-02156ff8636d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:59.687234 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a570398-18b8-46b3-8032-02156ff8636d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:59.687954 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a570398-18b8-46b3-8032-02156ff8636d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:59.689367 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a570398-18b8-46b3-8032-02156ff8636d None admin] RBAC: Adding query filter params (role.id=18c3c8a821b647f7982a955880bd0a9a, scope.project.id=214bc7d9ff5940989811937e08b70389, user.id=02a6d62d51854d3bb5bb2365bda8e8cb) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:59.690042 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a570398-18b8-46b3-8032-02156ff8636d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:59.692184 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a570398-18b8-46b3-8032-02156ff8636d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:59.699329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 106/205] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:36:59 2022] GET /identity/v3/role_assignments?user.id=02a6d62d51854d3bb5bb2365bda8e8cb&scope.project.id=214bc7d9ff5940989811937e08b70389&role.id=18c3c8a821b647f7982a955880bd0a9a => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:59.700631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a570398-18b8-46b3-8032-02156ff8636d None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:36:59.815148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ee0f05-07cf-455b-8ecf-a071010b4dbd None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:59.823470 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:36:59.894306 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0418394-5d11-45c3-a08d-0001cd8eb8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:59.895185 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0418394-5d11-45c3-a08d-0001cd8eb8ef None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:59.895582 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0418394-5d11-45c3-a08d-0001cd8eb8ef None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:59.895935 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0418394-5d11-45c3-a08d-0001cd8eb8ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:59.897149 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0418394-5d11-45c3-a08d-0001cd8eb8ef None admin] RBAC: Adding query filter params (role.id=0f660bd0ff4c4beda97ed7c24851d2d4, scope.project.id=e55f1cc9f9c9481698d6962c7af7fbfe, user.id=e0fac339988e453e8f6be2c7ba535863) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:36:59.897639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0418394-5d11-45c3-a08d-0001cd8eb8ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:59.900035 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0418394-5d11-45c3-a08d-0001cd8eb8ef None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:59.909425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a570398-18b8-46b3-8032-02156ff8636d None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:36:59.922432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 100/206] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:36:59 2022] GET /identity/v3/role_assignments?user.id=e0fac339988e453e8f6be2c7ba535863&scope.project.id=e55f1cc9f9c9481698d6962c7af7fbfe&role.id=0f660bd0ff4c4beda97ed7c24851d2d4 => generated 608 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:59.978850 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac703a20-1fb7-4ae5-9744-2075e41b6c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:36:59.979965 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac703a20-1fb7-4ae5-9744-2075e41b6c00 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:36:59.980435 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac703a20-1fb7-4ae5-9744-2075e41b6c00 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:36:59.980924 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac703a20-1fb7-4ae5-9744-2075e41b6c00 None admin] PATH_INFO: `/v3/projects/214bc7d9ff5940989811937e08b70389` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:36:59.982352 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac703a20-1fb7-4ae5-9744-2075e41b6c00 None admin] RBAC: Authorizing `identity:get_project(project_id=214bc7d9ff5940989811937e08b70389)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:36:59.984089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac703a20-1fb7-4ae5-9744-2075e41b6c00 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:36:59.991054 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 107/207] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:36:59 2022] GET /identity/v3/projects/214bc7d9ff5940989811937e08b70389 => generated 302 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:36:59.995728 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0418394-5d11-45c3-a08d-0001cd8eb8ef None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:00.071138 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:00.219098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0418394-5d11-45c3-a08d-0001cd8eb8ef None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:00.267018 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b546e-7b66-4e22-9909-292c7dcbe215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:00.267682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b546e-7b66-4e22-9909-292c7dcbe215 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:00.268030 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b546e-7b66-4e22-9909-292c7dcbe215 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:00.268311 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b546e-7b66-4e22-9909-292c7dcbe215 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:00.269541 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b546e-7b66-4e22-9909-292c7dcbe215 None admin] RBAC: Adding query filter params (role.id=6195dc21ae4749daa31797d2c42e050e, scope.project.id=214bc7d9ff5940989811937e08b70389, user.id=02a6d62d51854d3bb5bb2365bda8e8cb) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:00.270014 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b546e-7b66-4e22-9909-292c7dcbe215 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:00.271950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b546e-7b66-4e22-9909-292c7dcbe215 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:00.280302 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 101/208] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:36:59 2022] GET /identity/v3/role_assignments?user.id=02a6d62d51854d3bb5bb2365bda8e8cb&scope.project.id=214bc7d9ff5940989811937e08b70389&role.id=6195dc21ae4749daa31797d2c42e050e => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:00.801703 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02959029-b857-4646-919a-30a7786fdede None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:00.802498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02959029-b857-4646-919a-30a7786fdede None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:00.802626 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02959029-b857-4646-919a-30a7786fdede None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:00.804578 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 108/209] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:00.812946 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94d2ac-1b44-498c-aae2-b4b92cc75e89 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:00.813722 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94d2ac-1b44-498c-aae2-b4b92cc75e89 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:00.813722 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94d2ac-1b44-498c-aae2-b4b92cc75e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:00.840503 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-9e94d2ac-1b44-498c-aae2-b4b92cc75e89 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:00.841743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9e94d2ac-1b44-498c-aae2-b4b92cc75e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:00.950239 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 102/210] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:00.987308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa153f-897b-4f8c-8994-b3ae0c7e722e None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:00.988050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa153f-897b-4f8c-8994-b3ae0c7e722e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:00.988615 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa153f-897b-4f8c-8994-b3ae0c7e722e None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:00.991514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 109/211] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:01.000631 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae95dfc-09e2-4811-a9c3-27ba9224a0e8 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:01.001396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae95dfc-09e2-4811-a9c3-27ba9224a0e8 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:01.001987 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae95dfc-09e2-4811-a9c3-27ba9224a0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:01.029984 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-0ae95dfc-09e2-4811-a9c3-27ba9224a0e8 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:01.031659 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0ae95dfc-09e2-4811-a9c3-27ba9224a0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:01.066250 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:01.164734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 103/212] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:01.313844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa153f-897b-4f8c-8994-b3ae0c7e722e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:01.482709 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa153f-897b-4f8c-8994-b3ae0c7e722e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:01.519941 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:01.527281 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712b226-2f09-4bc0-920b-4cea5c77c895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:01.528284 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712b226-2f09-4bc0-920b-4cea5c77c895 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:01.528645 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712b226-2f09-4bc0-920b-4cea5c77c895 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:01.529128 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7712b226-2f09-4bc0-920b-4cea5c77c895 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:01.546448 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7712b226-2f09-4bc0-920b-4cea5c77c895 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:01.551629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-7712b226-2f09-4bc0-920b-4cea5c77c895 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:37:01.551629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:01.551629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:01.551629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:01.551629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:01.551629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:01.551629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:01.551629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:01.551629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:01.551629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:01.551629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:01.551629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:01.551629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:01.551629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:01.551629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:01.551629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:01.553229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:01.553229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:01.553229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:01.553229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:01.553229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:01.553229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:01.553229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:01.553229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:01.553229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:01.553229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:01.553229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:01.553229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:01.553229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:01.553229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:01.553229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:01.553229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:01.555843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:01.555843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:01.555843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:01.555843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:01.555843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:01.555843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:01.555843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:01.555843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:01.555843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:01.555843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:01.555843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:37:01.555843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:01.555843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 110/213] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:37:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:01.564769 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae95dfc-09e2-4811-a9c3-27ba9224a0e8 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:01.682988 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712b226-2f09-4bc0-920b-4cea5c77c895 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:01.709409 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae95dfc-09e2-4811-a9c3-27ba9224a0e8 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:01.756551 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee533b3-20b9-4f02-bb39-10c1c00a65f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:01.757313 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee533b3-20b9-4f02-bb39-10c1c00a65f2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:01.757617 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee533b3-20b9-4f02-bb39-10c1c00a65f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:01.757888 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee533b3-20b9-4f02-bb39-10c1c00a65f2 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:01.758608 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee533b3-20b9-4f02-bb39-10c1c00a65f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:01.758952 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee533b3-20b9-4f02-bb39-10c1c00a65f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:01.760607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee533b3-20b9-4f02-bb39-10c1c00a65f2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:01.772014 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 104/214] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:37:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:01.780910 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee533b3-20b9-4f02-bb39-10c1c00a65f2 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:01.807686 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:01.850032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7712b226-2f09-4bc0-920b-4cea5c77c895 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:01.894748 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ebdb1-c7e1-4779-abe7-3aee6f9e39fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:01.895575 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ebdb1-c7e1-4779-abe7-3aee6f9e39fd None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:01.895860 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ebdb1-c7e1-4779-abe7-3aee6f9e39fd None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:01.896143 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ebdb1-c7e1-4779-abe7-3aee6f9e39fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:01.897489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ebdb1-c7e1-4779-abe7-3aee6f9e39fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:01.899270 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ebdb1-c7e1-4779-abe7-3aee6f9e39fd None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:01.901292 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 111/215] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:01.907275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ebdb1-c7e1-4779-abe7-3aee6f9e39fd None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:01.922804 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee533b3-20b9-4f02-bb39-10c1c00a65f2 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:01.967592 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d078cd-9bca-4bad-bed6-f3ab9c6dac8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:01.968173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d078cd-9bca-4bad-bed6-f3ab9c6dac8e None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:01.968447 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d078cd-9bca-4bad-bed6-f3ab9c6dac8e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:01.968716 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d078cd-9bca-4bad-bed6-f3ab9c6dac8e None admin] PATH_INFO: `/v3/users/d50612dcae6e47188d3188d0a3ba4aed` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:01.986398 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d078cd-9bca-4bad-bed6-f3ab9c6dac8e None admin] RBAC: Authorizing `identity:get_user(user_id=d50612dcae6e47188d3188d0a3ba4aed)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:01.988463 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d078cd-9bca-4bad-bed6-f3ab9c6dac8e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:01.991086 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 105/216] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:37:01 2022] GET /identity/v3/users/d50612dcae6e47188d3188d0a3ba4aed => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:01.995188 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:01.998789 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d078cd-9bca-4bad-bed6-f3ab9c6dac8e None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:02.047575 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ebdb1-c7e1-4779-abe7-3aee6f9e39fd None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:02.087147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55ebb6-6d18-4a05-8d9c-29cd877cd3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:02.088062 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55ebb6-6d18-4a05-8d9c-29cd877cd3a6 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:02.088540 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55ebb6-6d18-4a05-8d9c-29cd877cd3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:02.088956 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55ebb6-6d18-4a05-8d9c-29cd877cd3a6 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:02.089787 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a55ebb6-6d18-4a05-8d9c-29cd877cd3a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:02.091486 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a55ebb6-6d18-4a05-8d9c-29cd877cd3a6 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:02.099262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:02.119248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 112/217] 10.222.0.71 () {60 vars in 1132 bytes} [Fri Aug 26 08:37:01 2022] POST /identity/v3/users => generated 297 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:37:02.147797 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d078cd-9bca-4bad-bed6-f3ab9c6dac8e None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:02.196676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fadae59-079e-4feb-b33a-5bda5b5a5adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:02.197386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fadae59-079e-4feb-b33a-5bda5b5a5adf None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:02.197726 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fadae59-079e-4feb-b33a-5bda5b5a5adf None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:02.197996 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fadae59-079e-4feb-b33a-5bda5b5a5adf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:02.284886 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fadae59-079e-4feb-b33a-5bda5b5a5adf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:02.286457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fadae59-079e-4feb-b33a-5bda5b5a5adf None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:02.367107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:02.407203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 106/218] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:37:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:37:02.413177 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55ebb6-6d18-4a05-8d9c-29cd877cd3a6 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:02.504891 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc88a8f-3629-4a66-bd3a-dbd0efa50fcc None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:02.505213 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc88a8f-3629-4a66-bd3a-dbd0efa50fcc None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:02.505457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc88a8f-3629-4a66-bd3a-dbd0efa50fcc None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:02.506850 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 107/219] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:37:02.513827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d356c-8bda-4fb8-b5f6-f46f26cc2559 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:02.514111 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d356c-8bda-4fb8-b5f6-f46f26cc2559 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:02.514366 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d356c-8bda-4fb8-b5f6-f46f26cc2559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:02.537670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b40d356c-8bda-4fb8-b5f6-f46f26cc2559 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:02.539588 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b40d356c-8bda-4fb8-b5f6-f46f26cc2559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:02.545478 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55ebb6-6d18-4a05-8d9c-29cd877cd3a6 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:02.573955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:02.600610 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55f87d-b4b6-4214-bf88-cc2d8aff6819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:02.601707 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55f87d-b4b6-4214-bf88-cc2d8aff6819 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:02.602238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55f87d-b4b6-4214-bf88-cc2d8aff6819 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:02.602626 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55f87d-b4b6-4214-bf88-cc2d8aff6819 None admin] PATH_INFO: `/v3/projects/214bc7d9ff5940989811937e08b70389` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:02.604944 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a55f87d-b4b6-4214-bf88-cc2d8aff6819 None admin] RBAC: Authorizing `identity:get_project(project_id=214bc7d9ff5940989811937e08b70389)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:02.607074 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a55f87d-b4b6-4214-bf88-cc2d8aff6819 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:02.609365 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 113/220] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:37:02 2022] GET /identity/v3/projects/214bc7d9ff5940989811937e08b70389 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:02.616159 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55f87d-b4b6-4214-bf88-cc2d8aff6819 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:02.650439 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 108/221] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:02.720091 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e1dbe-112b-4058-8253-85e9d2a0a822 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:02.720842 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e1dbe-112b-4058-8253-85e9d2a0a822 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:02.721304 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e1dbe-112b-4058-8253-85e9d2a0a822 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:02.724275 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 109/222] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:37:02.732133 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fe41f-8761-4f30-a4bc-47f7bb58d96a None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:02.732872 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fe41f-8761-4f30-a4bc-47f7bb58d96a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:02.733334 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fe41f-8761-4f30-a4bc-47f7bb58d96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:02.757951 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-623fe41f-8761-4f30-a4bc-47f7bb58d96a None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:02.760336 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-623fe41f-8761-4f30-a4bc-47f7bb58d96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:02.799940 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55f87d-b4b6-4214-bf88-cc2d8aff6819 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:02.882794 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca20b91-371e-4439-8284-2a2ee6556a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:02.884275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca20b91-371e-4439-8284-2a2ee6556a78 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:02.884734 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca20b91-371e-4439-8284-2a2ee6556a78 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:02.885224 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca20b91-371e-4439-8284-2a2ee6556a78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:02.886781 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca20b91-371e-4439-8284-2a2ee6556a78 None admin] RBAC: Adding query filter params (role.id=b7a48a25b0cd479d85fe5395d64c458e, scope.project.id=214bc7d9ff5940989811937e08b70389, user.id=d50612dcae6e47188d3188d0a3ba4aed) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:02.887489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca20b91-371e-4439-8284-2a2ee6556a78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:02.889338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca20b91-371e-4439-8284-2a2ee6556a78 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:02.897334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 114/223] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:37:02 2022] GET /identity/v3/role_assignments?user.id=d50612dcae6e47188d3188d0a3ba4aed&scope.project.id=214bc7d9ff5940989811937e08b70389&role.id=b7a48a25b0cd479d85fe5395d64c458e => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:02.907438 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 110/224] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 08:37:03.089932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca20b91-371e-4439-8284-2a2ee6556a78 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:03.157539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:03.251071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca20b91-371e-4439-8284-2a2ee6556a78 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:03.295270 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623fe41f-8761-4f30-a4bc-47f7bb58d96a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:03.341025 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b214f8c-aef9-46dc-bd37-c3aeb842b9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:03.342107 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b214f8c-aef9-46dc-bd37-c3aeb842b9a5 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:03.343251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b214f8c-aef9-46dc-bd37-c3aeb842b9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:03.343896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b214f8c-aef9-46dc-bd37-c3aeb842b9a5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:03.372730 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:03.373470 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b214f8c-aef9-46dc-bd37-c3aeb842b9a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:03.381490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-8b214f8c-aef9-46dc-bd37-c3aeb842b9a5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 08:37:03.381490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:03.381490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:03.381490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:03.381490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:03.381490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:03.381490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:03.381490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:03.381490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:03.381490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:03.381490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:03.381490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:03.381490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:03.381490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:03.381490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:03.381490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:03.382440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:03.382440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:03.382440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:03.382440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:03.382440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:03.382440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:03.382440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:03.382440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:03.382440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:03.382440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:03.382440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:03.382440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:03.382440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:03.382440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:03.382440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:03.382440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:03.383251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:03.383251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:03.383251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:03.383251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:03.383251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:03.383251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:03.383251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:03.383251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:03.383251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:03.383251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:03.383251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 08:37:03.383251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:03.395098 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 115/225] 10.222.0.71 () {56 vars in 1114 bytes} [Fri Aug 26 08:37:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:03.400668 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b214f8c-aef9-46dc-bd37-c3aeb842b9a5 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:03.486246 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623fe41f-8761-4f30-a4bc-47f7bb58d96a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:03.527096 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a08ff-c441-4229-a1ca-70a54b6bc884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:03.528208 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a08ff-c441-4229-a1ca-70a54b6bc884 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:03.528712 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a08ff-c441-4229-a1ca-70a54b6bc884 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:03.529173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a08ff-c441-4229-a1ca-70a54b6bc884 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:03.545928 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a08ff-c441-4229-a1ca-70a54b6bc884 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:03.552222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-9c7a08ff-c441-4229-a1ca-70a54b6bc884 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:37:03.552222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:03.552222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:03.552222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:03.552222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:03.552222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:03.552222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:03.552222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:03.552222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:03.552222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:03.552222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:03.552222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:03.552222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:03.552222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:03.552222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:03.552222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:03.553325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:03.553325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:03.553325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:03.553325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:03.553325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:03.553325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:03.553325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:03.553325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:03.553325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:03.553325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:03.553325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:03.553325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:03.553325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:03.553325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:03.553325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:03.553325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:03.554266 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:03.554266 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:03.554266 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:03.554266 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:03.554266 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:03.554266 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:03.554266 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:03.554266 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:03.554266 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:03.554266 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:03.554266 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:37:03.554266 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:03.556921 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 111/226] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:37:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:37:03.572190 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a08ff-c441-4229-a1ca-70a54b6bc884 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:03.610090 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b214f8c-aef9-46dc-bd37-c3aeb842b9a5 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:03.616112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:03.645255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73b8aa-b5d2-4c69-be96-dc4f8f9ac16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:03.645915 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73b8aa-b5d2-4c69-be96-dc4f8f9ac16e None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:03.646366 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73b8aa-b5d2-4c69-be96-dc4f8f9ac16e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:03.646654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73b8aa-b5d2-4c69-be96-dc4f8f9ac16e None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:03.648234 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73b8aa-b5d2-4c69-be96-dc4f8f9ac16e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:03.648234 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73b8aa-b5d2-4c69-be96-dc4f8f9ac16e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:03.649926 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73b8aa-b5d2-4c69-be96-dc4f8f9ac16e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:03.659443 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 116/227] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:37:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:37:03.672001 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73b8aa-b5d2-4c69-be96-dc4f8f9ac16e None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:03.726260 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:03.746053 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a08ff-c441-4229-a1ca-70a54b6bc884 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:03.804604 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb133a6-cb04-4e96-97ad-6ffedc85a265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:03.805594 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb133a6-cb04-4e96-97ad-6ffedc85a265 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:03.806114 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb133a6-cb04-4e96-97ad-6ffedc85a265 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:03.806632 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb133a6-cb04-4e96-97ad-6ffedc85a265 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:03.807741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb133a6-cb04-4e96-97ad-6ffedc85a265 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:03.808348 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb133a6-cb04-4e96-97ad-6ffedc85a265 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:03.810120 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb133a6-cb04-4e96-97ad-6ffedc85a265 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:03.827390 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 112/228] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:37:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:03.838968 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73b8aa-b5d2-4c69-be96-dc4f8f9ac16e None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:03.845303 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb133a6-cb04-4e96-97ad-6ffedc85a265 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:03.881658 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269a010-c427-4b9b-836b-c1fab874667d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:03.882220 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269a010-c427-4b9b-836b-c1fab874667d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:03.882496 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269a010-c427-4b9b-836b-c1fab874667d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:03.882774 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269a010-c427-4b9b-836b-c1fab874667d None admin] PATH_INFO: `/v3/users/02a6d62d51854d3bb5bb2365bda8e8cb` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:03.884353 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7269a010-c427-4b9b-836b-c1fab874667d None admin] RBAC: Authorizing `identity:get_user(user_id=02a6d62d51854d3bb5bb2365bda8e8cb)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:03.886223 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7269a010-c427-4b9b-836b-c1fab874667d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:03.887945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 117/229] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:37:03 2022] GET /identity/v3/users/02a6d62d51854d3bb5bb2365bda8e8cb => generated 283 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:03.897794 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269a010-c427-4b9b-836b-c1fab874667d None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:03.997198 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb133a6-cb04-4e96-97ad-6ffedc85a265 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:04.019095 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:04.089713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10e05e-c64f-40bd-93f8-e6c99261dc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:04.090643 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10e05e-c64f-40bd-93f8-e6c99261dc50 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:04.090936 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10e05e-c64f-40bd-93f8-e6c99261dc50 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:04.091236 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10e05e-c64f-40bd-93f8-e6c99261dc50 None admin] PATH_INFO: `/v3/users/02a6d62d51854d3bb5bb2365bda8e8cb` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:04.093671 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e10e05e-c64f-40bd-93f8-e6c99261dc50 None admin] RBAC: Authorizing `identity:get_user(user_id=02a6d62d51854d3bb5bb2365bda8e8cb)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:04.095224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e10e05e-c64f-40bd-93f8-e6c99261dc50 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:04.096902 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 113/230] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:37:03 2022] GET /identity/v3/users/02a6d62d51854d3bb5bb2365bda8e8cb => generated 283 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:04.105742 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10e05e-c64f-40bd-93f8-e6c99261dc50 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:04.156644 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:04.189621 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269a010-c427-4b9b-836b-c1fab874667d None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:04.274434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac17c8-3138-4ea1-9ddd-595c1361c15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:04.275550 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac17c8-3138-4ea1-9ddd-595c1361c15d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:04.276036 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac17c8-3138-4ea1-9ddd-595c1361c15d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:04.276495 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac17c8-3138-4ea1-9ddd-595c1361c15d None admin] PATH_INFO: `/v3/projects/214bc7d9ff5940989811937e08b70389` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:04.280976 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caac17c8-3138-4ea1-9ddd-595c1361c15d None admin] RBAC: Authorizing `identity:get_project(project_id=214bc7d9ff5940989811937e08b70389)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:04.282704 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caac17c8-3138-4ea1-9ddd-595c1361c15d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:04.290703 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 118/231] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:37:03 2022] GET /identity/v3/projects/214bc7d9ff5940989811937e08b70389 => generated 302 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:04.293920 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac17c8-3138-4ea1-9ddd-595c1361c15d None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:04.337575 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10e05e-c64f-40bd-93f8-e6c99261dc50 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:04.447443 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8f386-7e05-4218-b438-a77f67cf5f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:04.448377 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8f386-7e05-4218-b438-a77f67cf5f59 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:04.448797 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8f386-7e05-4218-b438-a77f67cf5f59 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:04.449106 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8f386-7e05-4218-b438-a77f67cf5f59 None admin] PATH_INFO: `/v3/projects/214bc7d9ff5940989811937e08b70389` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:04.456352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8f386-7e05-4218-b438-a77f67cf5f59 None admin] RBAC: Authorizing `identity:get_project(project_id=214bc7d9ff5940989811937e08b70389)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:04.457964 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d8f386-7e05-4218-b438-a77f67cf5f59 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:04.460058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 114/232] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:37:04 2022] GET /identity/v3/projects/214bc7d9ff5940989811937e08b70389 => generated 302 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:04.483340 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8f386-7e05-4218-b438-a77f67cf5f59 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:04.553896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac17c8-3138-4ea1-9ddd-595c1361c15d None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:04.573729 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:04.623298 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:04.638745 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8f386-7e05-4218-b438-a77f67cf5f59 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:04.688114 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e843e6-6fe2-4be4-bd44-b39c97ecfb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:04.689593 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e843e6-6fe2-4be4-bd44-b39c97ecfb12 None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:04.690043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af5cc4-2938-47ff-b0c1-8614eaed628a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:04.690252 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e843e6-6fe2-4be4-bd44-b39c97ecfb12 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:04.690752 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e843e6-6fe2-4be4-bd44-b39c97ecfb12 None admin] PATH_INFO: `/v3/projects/214bc7d9ff5940989811937e08b70389/users/02a6d62d51854d3bb5bb2365bda8e8cb/roles/6195dc21ae4749daa31797d2c42e050e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:04.691244 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af5cc4-2938-47ff-b0c1-8614eaed628a None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:04.691712 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af5cc4-2938-47ff-b0c1-8614eaed628a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:04.692181 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af5cc4-2938-47ff-b0c1-8614eaed628a None admin] PATH_INFO: `/v3/projects/214bc7d9ff5940989811937e08b70389/users/02a6d62d51854d3bb5bb2365bda8e8cb/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:04.694411 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af5cc4-2938-47ff-b0c1-8614eaed628a None admin] RBAC: Authorizing `identity:create_grant(project_id=214bc7d9ff5940989811937e08b70389, user_id=02a6d62d51854d3bb5bb2365bda8e8cb, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:04.696524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af5cc4-2938-47ff-b0c1-8614eaed628a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:04.700629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e843e6-6fe2-4be4-bd44-b39c97ecfb12 None admin] RBAC: Authorizing `identity:create_grant(project_id=214bc7d9ff5940989811937e08b70389, user_id=02a6d62d51854d3bb5bb2365bda8e8cb, role_id=6195dc21ae4749daa31797d2c42e050e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:04.703505 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e843e6-6fe2-4be4-bd44-b39c97ecfb12 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:04.731474 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 115/233] 10.222.0.71 () {58 vars in 1439 bytes} [Fri Aug 26 08:37:04 2022] PUT /identity/v3/projects/214bc7d9ff5940989811937e08b70389/users/02a6d62d51854d3bb5bb2365bda8e8cb/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:37:04.732935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 119/234] 10.222.0.71 () {58 vars in 1439 bytes} [Fri Aug 26 08:37:04 2022] PUT /identity/v3/projects/214bc7d9ff5940989811937e08b70389/users/02a6d62d51854d3bb5bb2365bda8e8cb/roles/6195dc21ae4749daa31797d2c42e050e => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:37:04.938935 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afba83-7dce-4fcd-b2bd-10f98437185e None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:04.939779 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afba83-7dce-4fcd-b2bd-10f98437185e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:04.940283 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afba83-7dce-4fcd-b2bd-10f98437185e None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:04.942485 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 116/235] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:37:04.950344 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcc494-6d1a-4f19-87e5-430390c5ae8f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:04.951475 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcc494-6d1a-4f19-87e5-430390c5ae8f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:04.952191 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcc494-6d1a-4f19-87e5-430390c5ae8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:04.978121 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-88dcc494-6d1a-4f19-87e5-430390c5ae8f None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:04.979956 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-88dcc494-6d1a-4f19-87e5-430390c5ae8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:05.081475 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 120/236] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 08:37:05.182909 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ae955-105d-44a5-b53c-09968c7bec66 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:05.183575 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ae955-105d-44a5-b53c-09968c7bec66 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:05.184051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ae955-105d-44a5-b53c-09968c7bec66 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:05.186097 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 117/237] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:05.192147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5885f-3f27-43f3-9978-de77e55e4350 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:05.192819 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5885f-3f27-43f3-9978-de77e55e4350 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:05.193296 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5885f-3f27-43f3-9978-de77e55e4350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:05.214934 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-41d5885f-3f27-43f3-9978-de77e55e4350 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:05.216575 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-41d5885f-3f27-43f3-9978-de77e55e4350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:05.310740 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 121/238] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:05.538474 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ae955-105d-44a5-b53c-09968c7bec66 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:05.675255 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ae955-105d-44a5-b53c-09968c7bec66 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:05.718137 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47e889-7be1-4a52-8aca-e760ccfb6288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:05.719337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47e889-7be1-4a52-8aca-e760ccfb6288 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:05.720139 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47e889-7be1-4a52-8aca-e760ccfb6288 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:05.720680 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47e889-7be1-4a52-8aca-e760ccfb6288 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:05.722593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a47e889-7be1-4a52-8aca-e760ccfb6288 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:05.724874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a47e889-7be1-4a52-8aca-e760ccfb6288 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:05.727008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 118/239] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:05.735972 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d5885f-3f27-43f3-9978-de77e55e4350 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:05.786194 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47e889-7be1-4a52-8aca-e760ccfb6288 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:05.787448 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:05.867621 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:05.934069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d5885f-3f27-43f3-9978-de77e55e4350 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:06.006594 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a8b87-d55d-4e5a-b3b0-ebf02bb62eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:06.007704 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a8b87-d55d-4e5a-b3b0-ebf02bb62eac None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:06.008219 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a8b87-d55d-4e5a-b3b0-ebf02bb62eac None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:06.008662 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a8b87-d55d-4e5a-b3b0-ebf02bb62eac None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:06.014008 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a8b87-d55d-4e5a-b3b0-ebf02bb62eac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:06.021797 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a8b87-d55d-4e5a-b3b0-ebf02bb62eac None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:06.067484 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 122/240] 10.222.0.71 () {60 vars in 1132 bytes} [Fri Aug 26 08:37:05 2022] POST /identity/v3/users => generated 293 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:37:06.099124 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95a572-3df9-4db5-87c9-be272fd7a6e2 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:06.099393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95a572-3df9-4db5-87c9-be272fd7a6e2 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:06.099706 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95a572-3df9-4db5-87c9-be272fd7a6e2 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:06.102672 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 123/241] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:06.107645 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf395a-629d-473a-8815-fe9068a540e6 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:06.108026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf395a-629d-473a-8815-fe9068a540e6 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:06.108240 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf395a-629d-473a-8815-fe9068a540e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:06.141842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-87cf395a-629d-473a-8815-fe9068a540e6 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:06.146385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47e889-7be1-4a52-8aca-e760ccfb6288 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:06.148850 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-87cf395a-629d-473a-8815-fe9068a540e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:06.284633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df1614-5027-4417-a687-6754690804ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:06.286286 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df1614-5027-4417-a687-6754690804ef None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:06.287160 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df1614-5027-4417-a687-6754690804ef None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:06.290150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df1614-5027-4417-a687-6754690804ef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:06.325571 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df1614-5027-4417-a687-6754690804ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:06.335549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-a9df1614-5027-4417-a687-6754690804ef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 08:37:06.335549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:06.335549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:06.335549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:06.335549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:06.335549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:06.335549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:06.335549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:06.335549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:06.335549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:06.335549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:06.335549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:06.335549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:06.335549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:06.335549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:06.335549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:06.337130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:06.337130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:06.337130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:06.337130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:06.337130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:06.337130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:06.337130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:06.337130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:06.337130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:06.337130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:06.337130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:06.337130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:06.337130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:06.337130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:06.337130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:06.337130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:06.338762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:06.338762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:06.338762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:06.338762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:06.338762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:06.338762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:06.338762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:06.338762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:06.338762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:06.338762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:06.338762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 08:37:06.338762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:06.338762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 119/242] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:37:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 551 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:37:06.345349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df1614-5027-4417-a687-6754690804ef None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:06.346028 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 124/243] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:06.457599 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:06.491784 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df1614-5027-4417-a687-6754690804ef None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:06.554269 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5443a-e9d2-4cc1-b7a6-3e17f96ce3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:06.555109 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5443a-e9d2-4cc1-b7a6-3e17f96ce3a6 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:06.555278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5443a-e9d2-4cc1-b7a6-3e17f96ce3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:06.555643 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5443a-e9d2-4cc1-b7a6-3e17f96ce3a6 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:06.556434 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5443a-e9d2-4cc1-b7a6-3e17f96ce3a6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:06.556743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5443a-e9d2-4cc1-b7a6-3e17f96ce3a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:06.559134 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5443a-e9d2-4cc1-b7a6-3e17f96ce3a6 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:06.570598 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 120/244] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:37:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:06.576574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf395a-629d-473a-8815-fe9068a540e6 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:06.601505 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:06.747567 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf395a-629d-473a-8815-fe9068a540e6 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:06.769096 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5443a-e9d2-4cc1-b7a6-3e17f96ce3a6 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:06.806392 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b3060-5230-47b7-89b0-73194f407a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:06.807170 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b3060-5230-47b7-89b0-73194f407a63 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:06.807446 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b3060-5230-47b7-89b0-73194f407a63 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:06.807730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b3060-5230-47b7-89b0-73194f407a63 None admin] PATH_INFO: `/v3/users/05fe99b921b34774950bf491dd68a7aa` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:06.840332 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b3060-5230-47b7-89b0-73194f407a63 None admin] RBAC: Authorizing `identity:get_user(user_id=05fe99b921b34774950bf491dd68a7aa)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:06.841846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b3060-5230-47b7-89b0-73194f407a63 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:06.844434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 125/245] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:37:06 2022] GET /identity/v3/users/05fe99b921b34774950bf491dd68a7aa => generated 297 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:06.852232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b3060-5230-47b7-89b0-73194f407a63 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:06.947830 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:06.950685 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5443a-e9d2-4cc1-b7a6-3e17f96ce3a6 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:07.009011 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b3060-5230-47b7-89b0-73194f407a63 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:07.013672 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caff1b5-8cc9-4bc8-949d-1f9d3b347329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:07.014381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caff1b5-8cc9-4bc8-949d-1f9d3b347329 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:07.014657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caff1b5-8cc9-4bc8-949d-1f9d3b347329 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:07.014967 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caff1b5-8cc9-4bc8-949d-1f9d3b347329 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:07.030184 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caff1b5-8cc9-4bc8-949d-1f9d3b347329 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:07.036506 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-3caff1b5-8cc9-4bc8-949d-1f9d3b347329 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:37:07.036506 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:07.036506 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:07.036506 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:07.036506 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:07.036506 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:07.036506 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:07.036506 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:07.036506 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:07.036506 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:07.036506 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:07.036506 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:07.036506 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:07.036506 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:07.036506 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:07.036506 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:07.037633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:07.037633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:07.037633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:07.037633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:07.037633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:07.037633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:07.037633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:07.037633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:07.037633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:07.037633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:07.037633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:07.037633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:07.037633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:07.037633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:07.037633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:07.037633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:07.038463 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:07.038463 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:07.038463 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:07.038463 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:07.038463 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:07.038463 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:07.038463 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:07.038463 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:07.038463 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:07.038463 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:07.038463 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:37:07.038463 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:07.038463 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 121/246] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:37:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:37:07.040721 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:07.047367 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caff1b5-8cc9-4bc8-949d-1f9d3b347329 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:07.061700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e7ec2-ddea-41f2-a64b-35a25d1e3f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:07.063119 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e7ec2-ddea-41f2-a64b-35a25d1e3f16 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:07.063843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e7ec2-ddea-41f2-a64b-35a25d1e3f16 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:07.064526 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e7ec2-ddea-41f2-a64b-35a25d1e3f16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:07.198691 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715e7ec2-ddea-41f2-a64b-35a25d1e3f16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:07.200590 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715e7ec2-ddea-41f2-a64b-35a25d1e3f16 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:07.317893 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caff1b5-8cc9-4bc8-949d-1f9d3b347329 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:07.391413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:07.450664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c2540f-df12-41ef-b43e-18184c783246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:07.451666 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2540f-df12-41ef-b43e-18184c783246 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:07.452134 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2540f-df12-41ef-b43e-18184c783246 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:07.452591 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2540f-df12-41ef-b43e-18184c783246 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:07.453695 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c2540f-df12-41ef-b43e-18184c783246 None admin] RBAC: Adding query filter params (name=member) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:07.454181 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c2540f-df12-41ef-b43e-18184c783246 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:07.459736 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c2540f-df12-41ef-b43e-18184c783246 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:07.464501 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 126/247] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:37:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 08:37:07.470779 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e7ec2-ddea-41f2-a64b-35a25d1e3f16 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:07.483555 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 122/248] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:37:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 437 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:07.494039 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c2540f-df12-41ef-b43e-18184c783246 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:07.685017 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:07.719388 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e7ec2-ddea-41f2-a64b-35a25d1e3f16 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:07.767134 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c2540f-df12-41ef-b43e-18184c783246 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:07.801691 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75411bb1-c4d3-4849-b3b0-22dd3e380655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:07.802913 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75411bb1-c4d3-4849-b3b0-22dd3e380655 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:07.804871 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75411bb1-c4d3-4849-b3b0-22dd3e380655 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:07.805552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75411bb1-c4d3-4849-b3b0-22dd3e380655 None admin] PATH_INFO: `/v3/projects/214bc7d9ff5940989811937e08b70389` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:07.808981 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75411bb1-c4d3-4849-b3b0-22dd3e380655 None admin] RBAC: Authorizing `identity:get_project(project_id=214bc7d9ff5940989811937e08b70389)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:07.814200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75411bb1-c4d3-4849-b3b0-22dd3e380655 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:07.816686 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 127/249] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:37:07 2022] GET /identity/v3/projects/214bc7d9ff5940989811937e08b70389 => generated 302 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:37:07.818430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f494e44-6473-433c-afa9-bede18f807fc None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:07.824583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f494e44-6473-433c-afa9-bede18f807fc None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:07.825305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f494e44-6473-433c-afa9-bede18f807fc None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:07.828512 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 128/250] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:07 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:07.832489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f494e44-6473-433c-afa9-bede18f807fc None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:07.859399 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd05d01-be84-42ac-8fa0-df38e2945123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:07.860379 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd05d01-be84-42ac-8fa0-df38e2945123 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:07.860877 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd05d01-be84-42ac-8fa0-df38e2945123 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:07.861324 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd05d01-be84-42ac-8fa0-df38e2945123 None admin] PATH_INFO: `/v3/users/d50612dcae6e47188d3188d0a3ba4aed` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:07.863734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd05d01-be84-42ac-8fa0-df38e2945123 None admin] RBAC: Authorizing `identity:get_user(user_id=d50612dcae6e47188d3188d0a3ba4aed)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:07.865767 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd05d01-be84-42ac-8fa0-df38e2945123 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:07.868967 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 123/251] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:37:07 2022] GET /identity/v3/users/d50612dcae6e47188d3188d0a3ba4aed => generated 297 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:07.873110 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f0b47-faa0-453b-96cc-f717bf09aaa3 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:07.877196 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f0b47-faa0-453b-96cc-f717bf09aaa3 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:07.877746 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f0b47-faa0-453b-96cc-f717bf09aaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:07.921476 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-5d0f0b47-faa0-453b-96cc-f717bf09aaa3 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:07.923468 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5d0f0b47-faa0-453b-96cc-f717bf09aaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:07.943630 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:08.050176 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f494e44-6473-433c-afa9-bede18f807fc None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:08.065077 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 124/252] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:08.066207 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f0b47-faa0-453b-96cc-f717bf09aaa3 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:08.122903 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d31ce5e-b6d7-4db8-8d9d-d5eed61e4a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:08.123846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31ce5e-b6d7-4db8-8d9d-d5eed61e4a9a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:08.124299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31ce5e-b6d7-4db8-8d9d-d5eed61e4a9a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:08.124740 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31ce5e-b6d7-4db8-8d9d-d5eed61e4a9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:08.126054 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d31ce5e-b6d7-4db8-8d9d-d5eed61e4a9a None admin] RBAC: Adding query filter params (role.id=0f660bd0ff4c4beda97ed7c24851d2d4, scope.project.id=214bc7d9ff5940989811937e08b70389, user.id=05fe99b921b34774950bf491dd68a7aa) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:08.126920 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d31ce5e-b6d7-4db8-8d9d-d5eed61e4a9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:08.129877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d31ce5e-b6d7-4db8-8d9d-d5eed61e4a9a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:08.152484 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 129/253] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:37:07 2022] GET /identity/v3/role_assignments?user.id=05fe99b921b34774950bf491dd68a7aa&scope.project.id=214bc7d9ff5940989811937e08b70389&role.id=0f660bd0ff4c4beda97ed7c24851d2d4 => generated 259 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:08.165898 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265f6c1-d11b-432d-a3d0-2e0c5c35fd34 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:08.167219 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265f6c1-d11b-432d-a3d0-2e0c5c35fd34 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:08.167848 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265f6c1-d11b-432d-a3d0-2e0c5c35fd34 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:08.169709 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 130/254] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:08 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:08.174920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:08.179555 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f623ed-2557-48e5-887e-8a736f00d135 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:08.179814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f623ed-2557-48e5-887e-8a736f00d135 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:08.180085 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f623ed-2557-48e5-887e-8a736f00d135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:08.223063 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f0b47-faa0-453b-96cc-f717bf09aaa3 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:08.248789 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-29f623ed-2557-48e5-887e-8a736f00d135 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:08.250538 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-29f623ed-2557-48e5-887e-8a736f00d135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:08.274014 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8c4e0-ba17-4183-8c85-c81ba1c6dd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:08.275770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8c4e0-ba17-4183-8c85-c81ba1c6dd15 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:08.276561 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8c4e0-ba17-4183-8c85-c81ba1c6dd15 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:08.277299 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8c4e0-ba17-4183-8c85-c81ba1c6dd15 None admin] PATH_INFO: `/v3/projects/214bc7d9ff5940989811937e08b70389` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:08.281165 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c8c4e0-ba17-4183-8c85-c81ba1c6dd15 None admin] RBAC: Authorizing `identity:get_project(project_id=214bc7d9ff5940989811937e08b70389)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:08.285996 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c8c4e0-ba17-4183-8c85-c81ba1c6dd15 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:08.290459 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 125/255] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:37:08 2022] GET /identity/v3/projects/214bc7d9ff5940989811937e08b70389 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:08.298985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8c4e0-ba17-4183-8c85-c81ba1c6dd15 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:08.536717 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 131/256] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 08:37:08.599297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8c4e0-ba17-4183-8c85-c81ba1c6dd15 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:08.622699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f623ed-2557-48e5-887e-8a736f00d135 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:08.660203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:08.675779 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763478d8-eecb-4fdf-a239-658d4510200e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:08.676808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763478d8-eecb-4fdf-a239-658d4510200e None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:08.677404 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763478d8-eecb-4fdf-a239-658d4510200e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:08.677722 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763478d8-eecb-4fdf-a239-658d4510200e None admin] PATH_INFO: `/v3/projects/214bc7d9ff5940989811937e08b70389/users/d50612dcae6e47188d3188d0a3ba4aed/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:08.681011 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763478d8-eecb-4fdf-a239-658d4510200e None admin] RBAC: Authorizing `identity:create_grant(project_id=214bc7d9ff5940989811937e08b70389, user_id=d50612dcae6e47188d3188d0a3ba4aed, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:08.683670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763478d8-eecb-4fdf-a239-658d4510200e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:08.719854 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 126/257] 10.222.0.71 () {58 vars in 1439 bytes} [Fri Aug 26 08:37:08 2022] PUT /identity/v3/projects/214bc7d9ff5940989811937e08b70389/users/d50612dcae6e47188d3188d0a3ba4aed/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:37:08.754740 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:08.942455 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f623ed-2557-48e5-887e-8a736f00d135 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:09.043451 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001e8f7-b9b3-43bb-9185-95a1e144927f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:09.044593 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001e8f7-b9b3-43bb-9185-95a1e144927f None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:09.045047 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001e8f7-b9b3-43bb-9185-95a1e144927f None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:09.045509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001e8f7-b9b3-43bb-9185-95a1e144927f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:09.075133 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4001e8f7-b9b3-43bb-9185-95a1e144927f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:09.084643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-4001e8f7-b9b3-43bb-9185-95a1e144927f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:37:09.084643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:09.084643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:09.084643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:09.084643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:09.084643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:09.084643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:09.084643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:09.084643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:09.084643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:09.084643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:09.084643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:09.084643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:09.084643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:09.084643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:09.084643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:09.086169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:09.086169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:09.086169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:09.086169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:09.086169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:09.086169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:09.086169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:09.086169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:09.086169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:09.086169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:09.086169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:09.086169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:09.086169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:09.086169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:09.086169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:09.086169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:09.087598 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:09.087598 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:09.087598 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:09.087598 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:09.087598 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:09.087598 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:09.087598 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:09.087598 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:09.087598 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:09.087598 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:09.087598 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:37:09.087598 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:09.087598 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 132/258] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:37:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 465 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:09.100426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763478d8-eecb-4fdf-a239-658d4510200e None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:09.227938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001e8f7-b9b3-43bb-9185-95a1e144927f None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:09.374998 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763478d8-eecb-4fdf-a239-658d4510200e None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:09.447058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb504b5-477b-4d57-9c83-e95662f904f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:09.447719 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb504b5-477b-4d57-9c83-e95662f904f1 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:09.447997 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb504b5-477b-4d57-9c83-e95662f904f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:09.448289 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb504b5-477b-4d57-9c83-e95662f904f1 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:09.449254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb504b5-477b-4d57-9c83-e95662f904f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:09.449581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb504b5-477b-4d57-9c83-e95662f904f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:09.451631 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb504b5-477b-4d57-9c83-e95662f904f1 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:09.467725 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 127/259] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:37:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:37:09.475867 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb504b5-477b-4d57-9c83-e95662f904f1 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:09.542447 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:09.565157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001e8f7-b9b3-43bb-9185-95a1e144927f None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:09.593515 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:09.673433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1884bec-9eda-4fae-8936-84475d33ee4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:09.674412 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1884bec-9eda-4fae-8936-84475d33ee4a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:09.679357 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1884bec-9eda-4fae-8936-84475d33ee4a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:09.679941 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1884bec-9eda-4fae-8936-84475d33ee4a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:09.699458 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb504b5-477b-4d57-9c83-e95662f904f1 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:09.708355 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1884bec-9eda-4fae-8936-84475d33ee4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:09.713654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-d1884bec-9eda-4fae-8936-84475d33ee4a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 08:37:09.713654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:09.713654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:09.713654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:09.713654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:09.713654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:09.713654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:09.713654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:09.713654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:09.713654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:09.713654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:09.713654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:09.713654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:09.713654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:09.713654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:09.713654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:09.715711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:09.715711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:09.715711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:09.715711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:09.715711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:09.715711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:09.715711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:09.715711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:09.715711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:09.715711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:09.715711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:09.715711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:09.715711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:09.715711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:09.715711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:09.715711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:09.717502 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:09.717502 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:09.717502 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:09.717502 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:09.717502 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:09.717502 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:09.717502 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:09.717502 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:09.717502 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:09.717502 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:09.717502 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 08:37:09.717502 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:09.719013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 133/260] 10.222.0.71 () {56 vars in 1114 bytes} [Fri Aug 26 08:37:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 488 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:37:09.724038 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1884bec-9eda-4fae-8936-84475d33ee4a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:09.775879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85348f0-f120-4cb9-8405-f8df9420d110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:09.777832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85348f0-f120-4cb9-8405-f8df9420d110 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:09.778490 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85348f0-f120-4cb9-8405-f8df9420d110 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:09.779173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85348f0-f120-4cb9-8405-f8df9420d110 None admin] PATH_INFO: `/v3/users/02a6d62d51854d3bb5bb2365bda8e8cb` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:09.781453 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85348f0-f120-4cb9-8405-f8df9420d110 None admin] RBAC: Authorizing `identity:get_user(user_id=02a6d62d51854d3bb5bb2365bda8e8cb)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:09.784495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85348f0-f120-4cb9-8405-f8df9420d110 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:09.788646 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 128/261] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:37:09 2022] GET /identity/v3/users/02a6d62d51854d3bb5bb2365bda8e8cb => generated 283 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:37:09.797116 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85348f0-f120-4cb9-8405-f8df9420d110 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:09.975266 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1884bec-9eda-4fae-8936-84475d33ee4a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:10.028428 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ced4c-4c26-46c8-9779-5f3256511630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:10.029057 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ced4c-4c26-46c8-9779-5f3256511630 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:10.029367 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ced4c-4c26-46c8-9779-5f3256511630 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:10.029594 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ced4c-4c26-46c8-9779-5f3256511630 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:10.030292 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ced4c-4c26-46c8-9779-5f3256511630 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:10.030583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ced4c-4c26-46c8-9779-5f3256511630 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:10.032031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ced4c-4c26-46c8-9779-5f3256511630 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:10.053431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85348f0-f120-4cb9-8405-f8df9420d110 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:10.057996 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 134/262] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:37:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:10.059342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca02a4-4460-4d45-8ea8-a72ed2d2f060 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:10.059630 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca02a4-4460-4d45-8ea8-a72ed2d2f060 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:10.059910 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca02a4-4460-4d45-8ea8-a72ed2d2f060 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:10.061228 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 135/263] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:10.071524 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dea925-1839-429a-b44c-8da624e566a5 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:10.071919 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dea925-1839-429a-b44c-8da624e566a5 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:10.072061 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dea925-1839-429a-b44c-8da624e566a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:10.095953 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-95dea925-1839-429a-b44c-8da624e566a5 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:10.097159 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-95dea925-1839-429a-b44c-8da624e566a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:10.111335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:10.118442 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:10.134758 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759fd2b-0c0e-45bc-8b5d-7e24e919e6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:10.135764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759fd2b-0c0e-45bc-8b5d-7e24e919e6db None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:10.136241 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759fd2b-0c0e-45bc-8b5d-7e24e919e6db None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:10.136742 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759fd2b-0c0e-45bc-8b5d-7e24e919e6db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:10.271155 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b759fd2b-0c0e-45bc-8b5d-7e24e919e6db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:10.272643 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b759fd2b-0c0e-45bc-8b5d-7e24e919e6db None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:10.277609 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 136/264] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:10.279386 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dea925-1839-429a-b44c-8da624e566a5 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:10.485680 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 129/265] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:37:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 690 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 08:37:10.504174 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759fd2b-0c0e-45bc-8b5d-7e24e919e6db None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:10.541908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dea925-1839-429a-b44c-8da624e566a5 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:10.551054 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:10.640299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea6eba0-fe29-4274-892a-4e9622aea228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:10.641383 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6eba0-fe29-4274-892a-4e9622aea228 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:10.644012 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6eba0-fe29-4274-892a-4e9622aea228 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:10.644561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea6eba0-fe29-4274-892a-4e9622aea228 None admin] PATH_INFO: `/v3/users/02a6d62d51854d3bb5bb2365bda8e8cb` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:10.647847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea6eba0-fe29-4274-892a-4e9622aea228 None admin] RBAC: Authorizing `identity:get_user(user_id=02a6d62d51854d3bb5bb2365bda8e8cb)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:10.649578 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea6eba0-fe29-4274-892a-4e9622aea228 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:10.653928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 137/266] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:37:10 2022] GET /identity/v3/users/02a6d62d51854d3bb5bb2365bda8e8cb => generated 283 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:10.657635 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eec03e-fd4f-4809-8a55-2a9f70ddd914 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:10.661779 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eec03e-fd4f-4809-8a55-2a9f70ddd914 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:10.662640 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eec03e-fd4f-4809-8a55-2a9f70ddd914 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:10.666303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 138/267] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:10 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:10.672679 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eec03e-fd4f-4809-8a55-2a9f70ddd914 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:10.714907 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:10.752350 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759fd2b-0c0e-45bc-8b5d-7e24e919e6db None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:10.803305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76dc07-d144-470b-807d-174a8abb0d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:10.805030 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76dc07-d144-470b-807d-174a8abb0d96 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:10.805584 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76dc07-d144-470b-807d-174a8abb0d96 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:10.806574 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76dc07-d144-470b-807d-174a8abb0d96 None admin] PATH_INFO: `/v3/projects/214bc7d9ff5940989811937e08b70389` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:10.808408 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb76dc07-d144-470b-807d-174a8abb0d96 None admin] RBAC: Authorizing `identity:get_project(project_id=214bc7d9ff5940989811937e08b70389)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:10.810120 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb76dc07-d144-470b-807d-174a8abb0d96 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:10.812735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 130/268] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:37:10 2022] GET /identity/v3/projects/214bc7d9ff5940989811937e08b70389 => generated 302 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:10.814445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91271132-bd68-4537-bd2f-f9e86810783d None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:10.814974 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91271132-bd68-4537-bd2f-f9e86810783d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:10.815454 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91271132-bd68-4537-bd2f-f9e86810783d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:10.839514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-91271132-bd68-4537-bd2f-f9e86810783d None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:10.841360 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-91271132-bd68-4537-bd2f-f9e86810783d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:10.894226 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eec03e-fd4f-4809-8a55-2a9f70ddd914 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:10.940054 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 131/269] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:10.946118 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91271132-bd68-4537-bd2f-f9e86810783d None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:10.963580 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:10.975756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067a23a-4112-40ac-83dd-d16a886b4260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:10.977387 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067a23a-4112-40ac-83dd-d16a886b4260 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:10.978507 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067a23a-4112-40ac-83dd-d16a886b4260 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:10.978982 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067a23a-4112-40ac-83dd-d16a886b4260 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:11.156345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067a23a-4112-40ac-83dd-d16a886b4260 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:11.159092 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067a23a-4112-40ac-83dd-d16a886b4260 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:11.191405 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:11.197723 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91271132-bd68-4537-bd2f-f9e86810783d None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:11.289302 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e498b9de-5543-49f3-9fa0-b57d7ac626a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:11.290359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498b9de-5543-49f3-9fa0-b57d7ac626a3 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:11.291239 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498b9de-5543-49f3-9fa0-b57d7ac626a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:11.291728 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498b9de-5543-49f3-9fa0-b57d7ac626a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:11.309425 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e498b9de-5543-49f3-9fa0-b57d7ac626a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:11.317431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-e498b9de-5543-49f3-9fa0-b57d7ac626a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:37:11.317431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:11.317431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:11.317431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:11.317431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:11.317431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:11.317431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:11.317431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:11.317431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:11.317431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:11.317431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:11.317431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:11.317431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:11.317431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:11.317431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:11.317431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:11.319069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:11.319069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:11.319069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:11.319069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:11.319069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:11.319069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:11.319069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:11.319069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:11.319069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:11.319069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:11.319069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:11.319069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:11.319069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:11.319069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:11.319069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:11.319069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:11.320435 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:11.320435 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:11.320435 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:11.320435 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:11.320435 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:11.320435 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:11.320435 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:11.320435 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:11.320435 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:11.320435 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:11.320435 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:37:11.320435 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:11.328456 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 132/270] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:37:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:11.328456 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e498b9de-5543-49f3-9fa0-b57d7ac626a3 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:11.379721 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 139/271] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:37:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 712 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:37:11.381769 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067a23a-4112-40ac-83dd-d16a886b4260 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:11.466355 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:11.495971 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e498b9de-5543-49f3-9fa0-b57d7ac626a3 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:11.554175 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067a23a-4112-40ac-83dd-d16a886b4260 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:11.577465 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:11.581637 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21875bd3-8c12-4b99-b716-b1e95b5bb59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:11.582291 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21875bd3-8c12-4b99-b716-b1e95b5bb59f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:11.582569 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21875bd3-8c12-4b99-b716-b1e95b5bb59f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:11.582846 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21875bd3-8c12-4b99-b716-b1e95b5bb59f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:11.583662 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21875bd3-8c12-4b99-b716-b1e95b5bb59f None admin] RBAC: Adding query filter params (role.id=18c3c8a821b647f7982a955880bd0a9a, scope.project.id=214bc7d9ff5940989811937e08b70389, user.id=02a6d62d51854d3bb5bb2365bda8e8cb) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:11.583980 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21875bd3-8c12-4b99-b716-b1e95b5bb59f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:11.585483 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21875bd3-8c12-4b99-b716-b1e95b5bb59f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:11.593218 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 133/272] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:37:11 2022] GET /identity/v3/role_assignments?user.id=02a6d62d51854d3bb5bb2365bda8e8cb&scope.project.id=214bc7d9ff5940989811937e08b70389&role.id=18c3c8a821b647f7982a955880bd0a9a => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:11.593969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21875bd3-8c12-4b99-b716-b1e95b5bb59f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:11.629307 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee4e96-e324-43bd-9814-6801486b9f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:11.629817 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee4e96-e324-43bd-9814-6801486b9f2d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:11.630093 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee4e96-e324-43bd-9814-6801486b9f2d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:11.630363 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee4e96-e324-43bd-9814-6801486b9f2d None admin] PATH_INFO: `/v3/domains/default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:11.631639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee4e96-e324-43bd-9814-6801486b9f2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:11.633139 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee4e96-e324-43bd-9814-6801486b9f2d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:11.634933 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 140/273] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:11.640275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee4e96-e324-43bd-9814-6801486b9f2d None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:11.765654 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21875bd3-8c12-4b99-b716-b1e95b5bb59f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:11.774409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee4e96-e324-43bd-9814-6801486b9f2d None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:11.811002 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f056f-6a9a-499e-94b7-5b6f1baa3b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:11.811733 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f056f-6a9a-499e-94b7-5b6f1baa3b59 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:11.811915 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f056f-6a9a-499e-94b7-5b6f1baa3b59 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:11.812166 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f056f-6a9a-499e-94b7-5b6f1baa3b59 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:11.812951 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f056f-6a9a-499e-94b7-5b6f1baa3b59 None admin] RBAC: Adding query filter params (name=member) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:11.813240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f056f-6a9a-499e-94b7-5b6f1baa3b59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:11.815159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f056f-6a9a-499e-94b7-5b6f1baa3b59 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:11.825075 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 134/274] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:37:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:11.825862 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f056f-6a9a-499e-94b7-5b6f1baa3b59 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:11.831530 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:11.837995 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1e47e-6fc5-4b8d-b4b2-23e15ea8be68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:11.838658 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1e47e-6fc5-4b8d-b4b2-23e15ea8be68 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:11.838965 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1e47e-6fc5-4b8d-b4b2-23e15ea8be68 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:11.839239 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1e47e-6fc5-4b8d-b4b2-23e15ea8be68 None admin] PATH_INFO: `/v3/projects/214bc7d9ff5940989811937e08b70389` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:11.840428 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1e47e-6fc5-4b8d-b4b2-23e15ea8be68 None admin] RBAC: Authorizing `identity:get_project(project_id=214bc7d9ff5940989811937e08b70389)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:11.841914 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1e47e-6fc5-4b8d-b4b2-23e15ea8be68 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:11.843721 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 141/275] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:37:11 2022] GET /identity/v3/projects/214bc7d9ff5940989811937e08b70389 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:11.844648 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1e47e-6fc5-4b8d-b4b2-23e15ea8be68 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:11.929764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:12.004771 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f056f-6a9a-499e-94b7-5b6f1baa3b59 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:12.041161 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1e47e-6fc5-4b8d-b4b2-23e15ea8be68 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:12.073690 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea0ee6-c6e9-40bc-8c12-c5055dc72c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:12.074313 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea0ee6-c6e9-40bc-8c12-c5055dc72c0a None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:12.075209 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea0ee6-c6e9-40bc-8c12-c5055dc72c0a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:12.075209 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea0ee6-c6e9-40bc-8c12-c5055dc72c0a None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:12.075635 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea0ee6-c6e9-40bc-8c12-c5055dc72c0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:12.077712 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea0ee6-c6e9-40bc-8c12-c5055dc72c0a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:12.121436 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f6735-8641-4dbb-b4d1-133e3bec8419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:12.122770 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f6735-8641-4dbb-b4d1-133e3bec8419 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:12.123257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f6735-8641-4dbb-b4d1-133e3bec8419 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:12.123775 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 135/276] 10.222.0.71 () {60 vars in 1132 bytes} [Fri Aug 26 08:37:11 2022] POST /identity/v3/users => generated 293 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:37:12.123881 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f6735-8641-4dbb-b4d1-133e3bec8419 None admin] PATH_INFO: `/v3/users/4aca1ecf02b4498184637dd534f3f157` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:12.125024 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea0ee6-c6e9-40bc-8c12-c5055dc72c0a None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:12.153657 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f6735-8641-4dbb-b4d1-133e3bec8419 None admin] RBAC: Authorizing `identity:get_user(user_id=4aca1ecf02b4498184637dd534f3f157)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:12.157595 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f6735-8641-4dbb-b4d1-133e3bec8419 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:12.162115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 142/277] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:37:11 2022] GET /identity/v3/users/4aca1ecf02b4498184637dd534f3f157 => generated 293 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:12.164473 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aaa435-dd36-45ef-9fe7-d49744ab5dce None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:12.164913 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aaa435-dd36-45ef-9fe7-d49744ab5dce None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:12.165410 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aaa435-dd36-45ef-9fe7-d49744ab5dce None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:12.175122 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 143/278] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:12.176241 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aaa435-dd36-45ef-9fe7-d49744ab5dce None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:12.229165 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:12.312213 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea0ee6-c6e9-40bc-8c12-c5055dc72c0a None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:12.349604 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aaa435-dd36-45ef-9fe7-d49744ab5dce None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:12.368774 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde6466-3559-436e-8770-c893c65b89bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:12.369828 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde6466-3559-436e-8770-c893c65b89bc None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:12.370297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde6466-3559-436e-8770-c893c65b89bc None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:12.370709 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde6466-3559-436e-8770-c893c65b89bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:12.372426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde6466-3559-436e-8770-c893c65b89bc None admin] RBAC: Adding query filter params (role.id=6195dc21ae4749daa31797d2c42e050e, scope.project.id=214bc7d9ff5940989811937e08b70389, user.id=02a6d62d51854d3bb5bb2365bda8e8cb) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:12.372750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde6466-3559-436e-8770-c893c65b89bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:12.373511 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:12.377195 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde6466-3559-436e-8770-c893c65b89bc None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:12.389424 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 136/279] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:37:12 2022] GET /identity/v3/role_assignments?user.id=02a6d62d51854d3bb5bb2365bda8e8cb&scope.project.id=214bc7d9ff5940989811937e08b70389&role.id=6195dc21ae4749daa31797d2c42e050e => generated 608 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:12.392188 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80261d-9924-4e72-ab5f-ce0daf02dc73 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:12.392755 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80261d-9924-4e72-ab5f-ce0daf02dc73 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:12.393261 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80261d-9924-4e72-ab5f-ce0daf02dc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:12.416088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2920d9-0fae-42be-9f8c-010237d26c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:12.417178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2920d9-0fae-42be-9f8c-010237d26c86 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:12.417831 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2920d9-0fae-42be-9f8c-010237d26c86 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:12.418328 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2920d9-0fae-42be-9f8c-010237d26c86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:12.419759 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2920d9-0fae-42be-9f8c-010237d26c86 None admin] RBAC: Adding query filter params (role.id=b7a48a25b0cd479d85fe5395d64c458e, user.id=4aca1ecf02b4498184637dd534f3f157, scope.system=all) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:12.420317 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2920d9-0fae-42be-9f8c-010237d26c86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:12.423376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2920d9-0fae-42be-9f8c-010237d26c86 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:12.424753 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-bd80261d-9924-4e72-ab5f-ce0daf02dc73 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:12.425995 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-bd80261d-9924-4e72-ab5f-ce0daf02dc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:12.437878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 144/280] 10.222.0.71 () {56 vars in 1308 bytes} [Fri Aug 26 08:37:12 2022] GET /identity/v3/role_assignments?user.id=4aca1ecf02b4498184637dd534f3f157&scope.system=all&role.id=b7a48a25b0cd479d85fe5395d64c458e => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:12.526960 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aaabcd-97d5-413d-bdd9-502d3c433594 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:12.527448 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aaabcd-97d5-413d-bdd9-502d3c433594 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:12.527809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aaabcd-97d5-413d-bdd9-502d3c433594 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:12.531904 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 145/281] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:37:12.535325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 137/282] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:12.536884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76beae05-edbd-4e64-9b80-c04597edaf4f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:12.537347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76beae05-edbd-4e64-9b80-c04597edaf4f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:12.537918 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76beae05-edbd-4e64-9b80-c04597edaf4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:12.563716 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-76beae05-edbd-4e64-9b80-c04597edaf4f None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:12.565654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-76beae05-edbd-4e64-9b80-c04597edaf4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:12.686479 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 146/283] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:12.980814 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80261d-9924-4e72-ab5f-ce0daf02dc73 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:12.991864 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76beae05-edbd-4e64-9b80-c04597edaf4f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:13.095308 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:13.228262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80261d-9924-4e72-ab5f-ce0daf02dc73 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:13.233180 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:13.237922 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76beae05-edbd-4e64-9b80-c04597edaf4f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:13.327624 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c151fff-c3a4-4ee7-9f2b-c2fd9bcd3a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:13.328740 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c151fff-c3a4-4ee7-9f2b-c2fd9bcd3a3c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:13.329231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c151fff-c3a4-4ee7-9f2b-c2fd9bcd3a3c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:13.329714 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c151fff-c3a4-4ee7-9f2b-c2fd9bcd3a3c None admin] PATH_INFO: `/v3/roles/member` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:13.333493 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5678e-c56f-403a-84ed-ad218a2a4783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:13.334617 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5678e-c56f-403a-84ed-ad218a2a4783 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:13.335158 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5678e-c56f-403a-84ed-ad218a2a4783 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:13.335678 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5678e-c56f-403a-84ed-ad218a2a4783 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:13.363310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5678e-c56f-403a-84ed-ad218a2a4783 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:13.366322 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c151fff-c3a4-4ee7-9f2b-c2fd9bcd3a3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:13.374495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-1c151fff-c3a4-4ee7-9f2b-c2fd9bcd3a3c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:37:13.374495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:13.374495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:13.374495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:13.374495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:13.374495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:13.374495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:13.374495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:13.374495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:13.374495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:13.374495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:13.374495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:13.374495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:13.374495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:13.374495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:13.374495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:13.376150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:13.376150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:13.376150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:13.376150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:13.376150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:13.376150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:13.376150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:13.376150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:13.376150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:13.376150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:13.376150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:13.376150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:13.376150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:13.376150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:13.376150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:13.376150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:13.377867 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:13.377867 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:13.377867 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:13.377867 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:13.377867 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:13.377867 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:13.377867 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:13.377867 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:13.377867 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:13.377867 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:13.377867 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:37:13.377867 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:13.377867 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 138/284] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:37:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:13.383385 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-c3d5678e-c56f-403a-84ed-ad218a2a4783 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 08:37:13.383385 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:13.383385 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:13.383385 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:13.383385 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:13.383385 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:13.383385 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:13.383385 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:13.383385 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:13.383385 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:13.383385 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:13.383385 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:13.383385 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:13.383385 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:13.383385 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:13.383385 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:13.386050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:13.386050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:13.386050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:13.386050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:13.386050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:13.386050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:13.386050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:13.386050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:13.386050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:13.386050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:13.386050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:13.386050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:13.386050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:13.386050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:13.386050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:13.386050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:13.388418 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:13.388418 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:13.388418 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:13.388418 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:13.388418 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:13.388418 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:13.388418 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:13.388418 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:13.388418 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:13.388418 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:13.388418 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 08:37:13.388418 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:13.388418 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 147/285] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:37:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:13.389979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c151fff-c3a4-4ee7-9f2b-c2fd9bcd3a3c None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:13.396407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5678e-c56f-403a-84ed-ad218a2a4783 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:13.636642 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5678e-c56f-403a-84ed-ad218a2a4783 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:13.653805 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c151fff-c3a4-4ee7-9f2b-c2fd9bcd3a3c None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:13.687281 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:13.715757 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a052943-7c16-44c2-8667-ce06053bb028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:13.717074 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a052943-7c16-44c2-8667-ce06053bb028 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:13.717566 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a052943-7c16-44c2-8667-ce06053bb028 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:13.718036 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a052943-7c16-44c2-8667-ce06053bb028 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:13.718971 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a052943-7c16-44c2-8667-ce06053bb028 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:13.719480 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a052943-7c16-44c2-8667-ce06053bb028 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:13.720596 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f8cd49-5b9c-4a8e-8822-5a1dd9ee38d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:13.721675 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a052943-7c16-44c2-8667-ce06053bb028 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:13.721783 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8cd49-5b9c-4a8e-8822-5a1dd9ee38d9 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:13.721989 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8cd49-5b9c-4a8e-8822-5a1dd9ee38d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:13.722261 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f8cd49-5b9c-4a8e-8822-5a1dd9ee38d9 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:13.723224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f8cd49-5b9c-4a8e-8822-5a1dd9ee38d9 None admin] RBAC: Adding query filter params (name=member) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:13.723543 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f8cd49-5b9c-4a8e-8822-5a1dd9ee38d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:13.725738 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f8cd49-5b9c-4a8e-8822-5a1dd9ee38d9 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:13.734015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 148/286] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:37:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:13.738839 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 139/287] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:37:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:13.741690 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a052943-7c16-44c2-8667-ce06053bb028 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:13.746040 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f8cd49-5b9c-4a8e-8822-5a1dd9ee38d9 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:13.788961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:13.898395 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a052943-7c16-44c2-8667-ce06053bb028 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:13.942176 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a84514-daec-41c9-b1b9-d6ed1f11c099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:13.943287 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a84514-daec-41c9-b1b9-d6ed1f11c099 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:13.943789 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a84514-daec-41c9-b1b9-d6ed1f11c099 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:13.944248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a84514-daec-41c9-b1b9-d6ed1f11c099 None admin] PATH_INFO: `/v3/users/05fe99b921b34774950bf491dd68a7aa` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:13.945729 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a84514-daec-41c9-b1b9-d6ed1f11c099 None admin] RBAC: Authorizing `identity:get_user(user_id=05fe99b921b34774950bf491dd68a7aa)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:13.947602 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a84514-daec-41c9-b1b9-d6ed1f11c099 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:13.949666 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 149/288] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:37:13 2022] GET /identity/v3/users/05fe99b921b34774950bf491dd68a7aa => generated 297 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:37:13.955885 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a84514-daec-41c9-b1b9-d6ed1f11c099 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:14.012958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f8cd49-5b9c-4a8e-8822-5a1dd9ee38d9 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:14.065941 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:14.078203 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a84514-daec-41c9-b1b9-d6ed1f11c099 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:14.119505 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab5e0c-406b-47c9-8f19-77206ff44a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:14.120534 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab5e0c-406b-47c9-8f19-77206ff44a83 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:14.121000 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab5e0c-406b-47c9-8f19-77206ff44a83 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:14.121458 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab5e0c-406b-47c9-8f19-77206ff44a83 None admin] PATH_INFO: `/v3/projects/214bc7d9ff5940989811937e08b70389` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:14.122845 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab5e0c-406b-47c9-8f19-77206ff44a83 None admin] RBAC: Authorizing `identity:get_project(project_id=214bc7d9ff5940989811937e08b70389)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:14.124741 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab5e0c-406b-47c9-8f19-77206ff44a83 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:14.126849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 150/289] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:37:13 2022] GET /identity/v3/projects/214bc7d9ff5940989811937e08b70389 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:14.130853 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb403b-0fa8-4f86-a830-d0222841b922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:14.131809 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb403b-0fa8-4f86-a830-d0222841b922 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:14.132275 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb403b-0fa8-4f86-a830-d0222841b922 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:14.132730 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb403b-0fa8-4f86-a830-d0222841b922 None admin] PATH_INFO: `/v3/users/d50612dcae6e47188d3188d0a3ba4aed` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:14.134131 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fb403b-0fa8-4f86-a830-d0222841b922 None admin] RBAC: Authorizing `identity:get_user(user_id=d50612dcae6e47188d3188d0a3ba4aed)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:14.136078 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab5e0c-406b-47c9-8f19-77206ff44a83 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:14.144036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fb403b-0fa8-4f86-a830-d0222841b922 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:14.149347 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 140/290] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:37:13 2022] GET /identity/v3/users/d50612dcae6e47188d3188d0a3ba4aed => generated 297 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:37:14.155733 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb403b-0fa8-4f86-a830-d0222841b922 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:14.253157 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:14.366741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb403b-0fa8-4f86-a830-d0222841b922 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:14.399076 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab5e0c-406b-47c9-8f19-77206ff44a83 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:14.462513 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b294b-5233-49ba-bdcc-dd01b486d3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:14.464212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b294b-5233-49ba-bdcc-dd01b486d3b0 None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:14.464965 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b294b-5233-49ba-bdcc-dd01b486d3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:14.465726 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b294b-5233-49ba-bdcc-dd01b486d3b0 None admin] PATH_INFO: `/v3/projects/214bc7d9ff5940989811937e08b70389/users/05fe99b921b34774950bf491dd68a7aa/roles/0f660bd0ff4c4beda97ed7c24851d2d4` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:14.469354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b294b-5233-49ba-bdcc-dd01b486d3b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=214bc7d9ff5940989811937e08b70389, user_id=05fe99b921b34774950bf491dd68a7aa, role_id=0f660bd0ff4c4beda97ed7c24851d2d4)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:14.472650 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b294b-5233-49ba-bdcc-dd01b486d3b0 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:14.480056 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5100998-93fe-499f-8ee0-7905813a5235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:14.480612 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5100998-93fe-499f-8ee0-7905813a5235 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:14.480894 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5100998-93fe-499f-8ee0-7905813a5235 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:14.481170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5100998-93fe-499f-8ee0-7905813a5235 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:14.509152 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 151/291] 10.222.0.71 () {58 vars in 1439 bytes} [Fri Aug 26 08:37:14 2022] PUT /identity/v3/projects/214bc7d9ff5940989811937e08b70389/users/05fe99b921b34774950bf491dd68a7aa/roles/0f660bd0ff4c4beda97ed7c24851d2d4 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:37:14.630674 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5100998-93fe-499f-8ee0-7905813a5235 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:14.634076 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5100998-93fe-499f-8ee0-7905813a5235 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:14.719613 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:14.795545 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 141/292] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:37:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 08:37:14.801430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b294b-5233-49ba-bdcc-dd01b486d3b0 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:14.824961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:14.913885 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4842994-13fe-4f76-aa9e-2e012d2604e7 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:14.914227 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4842994-13fe-4f76-aa9e-2e012d2604e7 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:14.914459 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4842994-13fe-4f76-aa9e-2e012d2604e7 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:14.916554 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 142/293] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:14.922671 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d358e-c03f-4b2d-8652-354d7f017ffd None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:14.923024 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d358e-c03f-4b2d-8652-354d7f017ffd None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:14.923269 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d358e-c03f-4b2d-8652-354d7f017ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:14.945994 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-a67d358e-c03f-4b2d-8652-354d7f017ffd None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:14.946218 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b294b-5233-49ba-bdcc-dd01b486d3b0 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:14.947804 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a67d358e-c03f-4b2d-8652-354d7f017ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:14.994331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e648512-7355-42c6-9a6a-c3152b06f8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:14.994954 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e648512-7355-42c6-9a6a-c3152b06f8c0 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:14.995268 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e648512-7355-42c6-9a6a-c3152b06f8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:14.995604 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e648512-7355-42c6-9a6a-c3152b06f8c0 None admin] PATH_INFO: `/v3/projects/214bc7d9ff5940989811937e08b70389` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:14.998340 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e648512-7355-42c6-9a6a-c3152b06f8c0 None admin] RBAC: Authorizing `identity:get_project(project_id=214bc7d9ff5940989811937e08b70389)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:15.000130 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e648512-7355-42c6-9a6a-c3152b06f8c0 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:15.002578 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 152/294] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:37:14 2022] GET /identity/v3/projects/214bc7d9ff5940989811937e08b70389 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:15.008918 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e648512-7355-42c6-9a6a-c3152b06f8c0 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:15.136822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 143/295] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 08:37:15.139069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d06368-c810-42f5-83b0-8bb1136bd709 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:15.139566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d06368-c810-42f5-83b0-8bb1136bd709 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:15.140042 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d06368-c810-42f5-83b0-8bb1136bd709 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:15.142869 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 144/296] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:15.154106 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ba2ed-f59f-485a-99cd-c32a0765933f None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:15.154690 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ba2ed-f59f-485a-99cd-c32a0765933f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:15.155328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ba2ed-f59f-485a-99cd-c32a0765933f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:15.178819 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:15.193243 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-257ba2ed-f59f-485a-99cd-c32a0765933f None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:15.195701 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-257ba2ed-f59f-485a-99cd-c32a0765933f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:15.209954 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e648512-7355-42c6-9a6a-c3152b06f8c0 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:15.293716 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef59f68-5fb2-4289-acbb-4fea18da78cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:15.294641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef59f68-5fb2-4289-acbb-4fea18da78cf None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:15.295091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef59f68-5fb2-4289-acbb-4fea18da78cf None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:15.295502 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef59f68-5fb2-4289-acbb-4fea18da78cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:15.296406 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef59f68-5fb2-4289-acbb-4fea18da78cf None admin] RBAC: Adding query filter params (role.id=b7a48a25b0cd479d85fe5395d64c458e, scope.project.id=214bc7d9ff5940989811937e08b70389, user.id=d50612dcae6e47188d3188d0a3ba4aed) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:15.296873 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef59f68-5fb2-4289-acbb-4fea18da78cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:15.298528 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef59f68-5fb2-4289-acbb-4fea18da78cf None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:15.306759 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 153/297] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:37:15 2022] GET /identity/v3/role_assignments?user.id=d50612dcae6e47188d3188d0a3ba4aed&scope.project.id=214bc7d9ff5940989811937e08b70389&role.id=b7a48a25b0cd479d85fe5395d64c458e => generated 608 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:37:15.356868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 145/298] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:15.498796 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d784825-6b1b-4584-982c-49001d306f9d None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:15.500038 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d784825-6b1b-4584-982c-49001d306f9d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:15.500529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d784825-6b1b-4584-982c-49001d306f9d None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:15.502815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 154/299] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:15.509951 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee207e-df15-474d-b574-33783c7922cf None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:15.510751 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee207e-df15-474d-b574-33783c7922cf None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:15.511529 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee207e-df15-474d-b574-33783c7922cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:15.521510 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:15.541519 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-bbee207e-df15-474d-b574-33783c7922cf None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:15.542815 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-bbee207e-df15-474d-b574-33783c7922cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:15.562594 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d784825-6b1b-4584-982c-49001d306f9d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:15.656871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 146/300] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:15.738489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d784825-6b1b-4584-982c-49001d306f9d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:15.781975 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab0781-a4df-4ea5-ab43-22dc25cd21f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:15.782641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab0781-a4df-4ea5-ab43-22dc25cd21f2 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:15.783027 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab0781-a4df-4ea5-ab43-22dc25cd21f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:15.783314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab0781-a4df-4ea5-ab43-22dc25cd21f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:15.797869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dab0781-a4df-4ea5-ab43-22dc25cd21f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:15.801795 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee207e-df15-474d-b574-33783c7922cf None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:15.804952 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-1dab0781-a4df-4ea5-ab43-22dc25cd21f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:37:15.804952 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:15.804952 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:15.804952 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:15.804952 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:15.804952 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:15.804952 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:15.804952 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:15.804952 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:15.804952 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:15.804952 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:15.804952 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:15.804952 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:15.804952 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:15.804952 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:15.804952 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:15.805863 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:15.805863 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:15.805863 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:15.805863 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:15.805863 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:15.805863 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:15.805863 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:15.805863 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:15.805863 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:15.805863 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:15.805863 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:15.805863 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:15.805863 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:15.805863 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:15.805863 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:15.805863 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:15.806649 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:15.806649 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:15.806649 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:15.806649 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:15.806649 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:15.806649 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:15.806649 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:15.806649 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:15.806649 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:15.806649 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:15.806649 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:37:15.806649 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:15.807991 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 155/301] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:37:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:15.819396 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab0781-a4df-4ea5-ab43-22dc25cd21f2 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:15.835567 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:15.965281 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab0781-a4df-4ea5-ab43-22dc25cd21f2 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:15.967523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee207e-df15-474d-b574-33783c7922cf None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:16.000531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:16.006919 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b322a2-6443-4016-8235-777673458ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:16.007253 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fab752-1636-4fa0-ace2-deba1a787086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:16.007560 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fab752-1636-4fa0-ace2-deba1a787086 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:16.007719 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b322a2-6443-4016-8235-777673458ca9 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:16.007883 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b322a2-6443-4016-8235-777673458ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:16.008030 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fab752-1636-4fa0-ace2-deba1a787086 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:16.008167 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b322a2-6443-4016-8235-777673458ca9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:16.008302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fab752-1636-4fa0-ace2-deba1a787086 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:16.009036 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fab752-1636-4fa0-ace2-deba1a787086 None admin] RBAC: Adding query filter params (name=member) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:16.009388 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fab752-1636-4fa0-ace2-deba1a787086 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:16.011061 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fab752-1636-4fa0-ace2-deba1a787086 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:16.020398 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b322a2-6443-4016-8235-777673458ca9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:16.021168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 156/302] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:37:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:16.022049 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fab752-1636-4fa0-ace2-deba1a787086 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:16.030022 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-21b322a2-6443-4016-8235-777673458ca9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 08:37:16.030022 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:16.030022 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:16.030022 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:16.030022 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:16.030022 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:16.030022 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:16.030022 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:16.030022 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:16.030022 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:16.030022 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:16.030022 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:16.030022 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:16.030022 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:16.030022 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:16.030022 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:16.031108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:16.031108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:16.031108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:16.031108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:16.031108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:16.031108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:16.031108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:16.031108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:16.031108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:16.031108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:16.031108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:16.031108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:16.031108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:16.031108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:16.031108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:16.031108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:16.031899 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:16.031899 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:16.031899 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:16.031899 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:16.031899 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:16.031899 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:16.031899 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:16.031899 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:16.031899 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:16.031899 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:16.031899 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 08:37:16.031899 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:16.031899 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 147/303] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:37:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:16.032727 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b322a2-6443-4016-8235-777673458ca9 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:16.125812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:16.159440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fab752-1636-4fa0-ace2-deba1a787086 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:16.198794 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b322a2-6443-4016-8235-777673458ca9 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:16.213752 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5643089-23be-4bba-aff2-03fddb329741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:16.214502 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5643089-23be-4bba-aff2-03fddb329741 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:16.214968 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5643089-23be-4bba-aff2-03fddb329741 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:16.215671 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5643089-23be-4bba-aff2-03fddb329741 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:16.217960 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5643089-23be-4bba-aff2-03fddb329741 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:16.219457 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5643089-23be-4bba-aff2-03fddb329741 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:16.222216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 157/304] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:16.223698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5643089-23be-4bba-aff2-03fddb329741 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:16.264591 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466e234-bfcc-4f98-95a8-43f76d2181f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:16.265238 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466e234-bfcc-4f98-95a8-43f76d2181f9 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:16.265524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466e234-bfcc-4f98-95a8-43f76d2181f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:16.265780 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466e234-bfcc-4f98-95a8-43f76d2181f9 None admin] PATH_INFO: `/v3/users/4aca1ecf02b4498184637dd534f3f157` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:16.266986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7466e234-bfcc-4f98-95a8-43f76d2181f9 None admin] RBAC: Authorizing `identity:get_user(user_id=4aca1ecf02b4498184637dd534f3f157)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:16.268507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7466e234-bfcc-4f98-95a8-43f76d2181f9 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:16.270479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 148/305] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:37:16 2022] GET /identity/v3/users/4aca1ecf02b4498184637dd534f3f157 => generated 293 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:16.271084 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466e234-bfcc-4f98-95a8-43f76d2181f9 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:16.387210 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:16.391133 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5643089-23be-4bba-aff2-03fddb329741 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:16.436653 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ecf72-b73a-418c-9cda-bb81c9fc0db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:16.437281 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ecf72-b73a-418c-9cda-bb81c9fc0db6 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:16.437550 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ecf72-b73a-418c-9cda-bb81c9fc0db6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:16.437818 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ecf72-b73a-418c-9cda-bb81c9fc0db6 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:16.438481 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ecf72-b73a-418c-9cda-bb81c9fc0db6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:16.439433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ecf72-b73a-418c-9cda-bb81c9fc0db6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:16.441320 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ecf72-b73a-418c-9cda-bb81c9fc0db6 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:16.454965 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 158/306] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:37:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:16.454965 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ecf72-b73a-418c-9cda-bb81c9fc0db6 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:16.454965 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:16.465169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466e234-bfcc-4f98-95a8-43f76d2181f9 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:16.520879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e472d9-31e9-468d-aaf7-df4d6bccf703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:16.521954 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e472d9-31e9-468d-aaf7-df4d6bccf703 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:16.522453 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e472d9-31e9-468d-aaf7-df4d6bccf703 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:16.522947 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e472d9-31e9-468d-aaf7-df4d6bccf703 None admin] PATH_INFO: `/v3/groups` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:16.524167 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e472d9-31e9-468d-aaf7-df4d6bccf703 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:16.526990 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e472d9-31e9-468d-aaf7-df4d6bccf703 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:16.549850 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 149/307] 10.222.0.71 () {60 vars in 1134 bytes} [Fri Aug 26 08:37:16 2022] POST /identity/v3/groups => generated 229 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:37:16.555186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e472d9-31e9-468d-aaf7-df4d6bccf703 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:16.603512 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ecf72-b73a-418c-9cda-bb81c9fc0db6 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:16.647694 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae239de-5bd3-4ed0-9fe5-4b65f0392b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:16.648736 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae239de-5bd3-4ed0-9fe5-4b65f0392b21 None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:16.649140 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae239de-5bd3-4ed0-9fe5-4b65f0392b21 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:16.649499 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae239de-5bd3-4ed0-9fe5-4b65f0392b21 None admin] PATH_INFO: `/v3/system/users/4aca1ecf02b4498184637dd534f3f157/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:16.651676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae239de-5bd3-4ed0-9fe5-4b65f0392b21 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4aca1ecf02b4498184637dd534f3f157, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:16.654417 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae239de-5bd3-4ed0-9fe5-4b65f0392b21 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:16.670991 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 159/308] 10.222.0.71 () {58 vars in 1334 bytes} [Fri Aug 26 08:37:16 2022] PUT /identity/v3/system/users/4aca1ecf02b4498184637dd534f3f157/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:37:16.723169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e472d9-31e9-468d-aaf7-df4d6bccf703 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:16.759496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:16.778734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b18131-10b3-4eb8-bd50-c3544a850fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:16.779441 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b18131-10b3-4eb8-bd50-c3544a850fed None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:16.780173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b18131-10b3-4eb8-bd50-c3544a850fed None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:16.780448 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b18131-10b3-4eb8-bd50-c3544a850fed None admin] PATH_INFO: `/v3/users/afa2909001ab42ab8b35c9f6a894dd15` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:16.804447 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b18131-10b3-4eb8-bd50-c3544a850fed None admin] RBAC: Authorizing `identity:get_user(user_id=afa2909001ab42ab8b35c9f6a894dd15)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:16.804447 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b18131-10b3-4eb8-bd50-c3544a850fed None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:16.804447 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 150/309] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:37:16 2022] GET /identity/v3/users/afa2909001ab42ab8b35c9f6a894dd15 => generated 293 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:16.811248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae239de-5bd3-4ed0-9fe5-4b65f0392b21 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:16.917443 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:16.970998 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae239de-5bd3-4ed0-9fe5-4b65f0392b21 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:16.985911 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae109f3-e62e-440e-b6d3-2051560300b9 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:16.986377 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae109f3-e62e-440e-b6d3-2051560300b9 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:16.986480 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae109f3-e62e-440e-b6d3-2051560300b9 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:16.991052 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 151/310] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:16.996676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cca98-c7ed-4012-b92e-610e930c25e0 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:16.996970 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cca98-c7ed-4012-b92e-610e930c25e0 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:16.997220 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cca98-c7ed-4012-b92e-610e930c25e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:17.018614 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ce3cca98-c7ed-4012-b92e-610e930c25e0 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:17.019027 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59532b75-a4b8-4160-88b2-8a82f34f6877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:17.019878 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ce3cca98-c7ed-4012-b92e-610e930c25e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:17.020057 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59532b75-a4b8-4160-88b2-8a82f34f6877 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:17.020545 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59532b75-a4b8-4160-88b2-8a82f34f6877 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:17.021038 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59532b75-a4b8-4160-88b2-8a82f34f6877 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:17.022867 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59532b75-a4b8-4160-88b2-8a82f34f6877 None admin] RBAC: Adding query filter params (role.id=0f660bd0ff4c4beda97ed7c24851d2d4, user.id=afa2909001ab42ab8b35c9f6a894dd15, scope.system=all) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:17.024860 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59532b75-a4b8-4160-88b2-8a82f34f6877 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:17.028297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59532b75-a4b8-4160-88b2-8a82f34f6877 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:17.043448 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 160/311] 10.222.0.71 () {56 vars in 1308 bytes} [Fri Aug 26 08:37:16 2022] GET /identity/v3/role_assignments?user.id=afa2909001ab42ab8b35c9f6a894dd15&scope.system=all&role.id=0f660bd0ff4c4beda97ed7c24851d2d4 => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:37:17.120646 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 152/312] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:17.392985 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59532b75-a4b8-4160-88b2-8a82f34f6877 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:17.505882 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59532b75-a4b8-4160-88b2-8a82f34f6877 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:17.539314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:17.547115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55153049-8075-47ef-95a6-e6181681cf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:17.548409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55153049-8075-47ef-95a6-e6181681cf38 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:17.548738 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55153049-8075-47ef-95a6-e6181681cf38 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:17.549109 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55153049-8075-47ef-95a6-e6181681cf38 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:17.562824 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55153049-8075-47ef-95a6-e6181681cf38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:17.568673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-55153049-8075-47ef-95a6-e6181681cf38 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 08:37:17.568673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:17.568673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:17.568673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:17.568673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:17.568673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:17.568673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:17.568673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:17.568673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:17.568673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:17.568673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:17.568673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:17.568673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:17.568673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:17.568673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:17.568673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:17.570409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:17.570409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:17.570409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:17.570409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:17.570409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:17.570409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:17.570409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:17.570409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:17.570409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:17.570409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:17.570409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:17.570409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:17.570409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:17.570409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:17.570409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:17.570409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:17.571900 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:17.571900 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:17.571900 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:17.571900 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:17.571900 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:17.571900 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:17.571900 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:17.571900 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:17.571900 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:17.571900 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:17.571900 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 08:37:17.571900 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:17.574786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 161/313] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:37:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:17.576336 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cca98-c7ed-4012-b92e-610e930c25e0 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:17.698480 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cca98-c7ed-4012-b92e-610e930c25e0 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:17.710480 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:17.742231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde79cd1-d893-452f-b886-4138f7befdcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:17.742870 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde79cd1-d893-452f-b886-4138f7befdcc None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:17.743170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde79cd1-d893-452f-b886-4138f7befdcc None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:17.743441 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde79cd1-d893-452f-b886-4138f7befdcc None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:17.744128 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde79cd1-d893-452f-b886-4138f7befdcc None admin] RBAC: Adding query filter params (name=reader) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:17.744430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde79cd1-d893-452f-b886-4138f7befdcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:17.745850 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde79cd1-d893-452f-b886-4138f7befdcc None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:17.760817 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 153/314] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:37:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:17.762225 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55153049-8075-47ef-95a6-e6181681cf38 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:17.879367 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55153049-8075-47ef-95a6-e6181681cf38 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:17.920150 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ddbce0-b875-4fed-be1e-04ac64726c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:17.920806 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ddbce0-b875-4fed-be1e-04ac64726c83 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:17.921083 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ddbce0-b875-4fed-be1e-04ac64726c83 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:17.921357 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ddbce0-b875-4fed-be1e-04ac64726c83 None admin] PATH_INFO: `/v3/users/05fe99b921b34774950bf491dd68a7aa` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:17.922620 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ddbce0-b875-4fed-be1e-04ac64726c83 None admin] RBAC: Authorizing `identity:get_user(user_id=05fe99b921b34774950bf491dd68a7aa)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:17.924250 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ddbce0-b875-4fed-be1e-04ac64726c83 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:17.925987 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 162/315] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:37:17 2022] GET /identity/v3/users/05fe99b921b34774950bf491dd68a7aa => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:17.932069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde79cd1-d893-452f-b886-4138f7befdcc None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:18.056010 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde79cd1-d893-452f-b886-4138f7befdcc None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:18.095971 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774a511-dffd-4c27-8657-8493535cf8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:18.096603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774a511-dffd-4c27-8657-8493535cf8ab None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:18.096880 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774a511-dffd-4c27-8657-8493535cf8ab None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:18.097151 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774a511-dffd-4c27-8657-8493535cf8ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:18.142860 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:18.177904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7774a511-dffd-4c27-8657-8493535cf8ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:18.179455 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7774a511-dffd-4c27-8657-8493535cf8ab None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:18.298601 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 154/316] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:37:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:37:18.305242 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ddbce0-b875-4fed-be1e-04ac64726c83 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:18.377098 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:18.428265 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ddbce0-b875-4fed-be1e-04ac64726c83 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:18.467472 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcbb9f3-8e7e-440e-ba7d-132771de034a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:18.468222 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbb9f3-8e7e-440e-ba7d-132771de034a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:18.468509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbb9f3-8e7e-440e-ba7d-132771de034a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:18.468784 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbb9f3-8e7e-440e-ba7d-132771de034a None admin] PATH_INFO: `/v3/projects/214bc7d9ff5940989811937e08b70389` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:18.470087 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcbb9f3-8e7e-440e-ba7d-132771de034a None admin] RBAC: Authorizing `identity:get_project(project_id=214bc7d9ff5940989811937e08b70389)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:18.471689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcbb9f3-8e7e-440e-ba7d-132771de034a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:18.473474 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 163/317] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:37:18 2022] GET /identity/v3/projects/214bc7d9ff5940989811937e08b70389 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:18.481047 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774a511-dffd-4c27-8657-8493535cf8ab None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:18.562924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:18.599431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774a511-dffd-4c27-8657-8493535cf8ab None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:18.638005 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5cd981-1861-4a08-b90b-727637e53b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:18.638624 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cd981-1861-4a08-b90b-727637e53b36 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:18.638914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cd981-1861-4a08-b90b-727637e53b36 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:18.639189 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cd981-1861-4a08-b90b-727637e53b36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:18.639939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5cd981-1861-4a08-b90b-727637e53b36 None admin] RBAC: Adding query filter params (role.id=0f660bd0ff4c4beda97ed7c24851d2d4, scope.project.id=214bc7d9ff5940989811937e08b70389, user.id=05fe99b921b34774950bf491dd68a7aa) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:18.640242 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5cd981-1861-4a08-b90b-727637e53b36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:18.641780 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5cd981-1861-4a08-b90b-727637e53b36 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:18.655100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 155/318] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:37:18 2022] GET /identity/v3/role_assignments?user.id=05fe99b921b34774950bf491dd68a7aa&scope.project.id=214bc7d9ff5940989811937e08b70389&role.id=0f660bd0ff4c4beda97ed7c24851d2d4 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:18.934190 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2165f-98e4-42a8-a834-9a3160eb0f27 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:18.934877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2165f-98e4-42a8-a834-9a3160eb0f27 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:18.934877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f2165f-98e4-42a8-a834-9a3160eb0f27 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:18.936518 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 164/319] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:18.943111 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63519a-1383-4830-b129-d134e74ac2c7 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:18.943692 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63519a-1383-4830-b129-d134e74ac2c7 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:18.944267 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63519a-1383-4830-b129-d134e74ac2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:18.948641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4ef8e-f9b8-4eb6-a5ba-2a7596e53adb None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:18.948938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4ef8e-f9b8-4eb6-a5ba-2a7596e53adb None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:18.949192 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4ef8e-f9b8-4eb6-a5ba-2a7596e53adb None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:18.950814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 165/320] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:37:18.956804 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f0ee9-4885-4d6d-9d19-0f115c2c0674 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:18.957144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f0ee9-4885-4d6d-9d19-0f115c2c0674 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:18.957350 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f0ee9-4885-4d6d-9d19-0f115c2c0674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:18.968946 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-fe63519a-1383-4830-b129-d134e74ac2c7 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:18.970686 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-fe63519a-1383-4830-b129-d134e74ac2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:18.976933 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-064f0ee9-4885-4d6d-9d19-0f115c2c0674 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:18.978217 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-064f0ee9-4885-4d6d-9d19-0f115c2c0674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:19.079489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 166/321] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:19.098988 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 156/322] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:19.438514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f0ee9-4885-4d6d-9d19-0f115c2c0674 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:19.530537 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:19.537150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63519a-1383-4830-b129-d134e74ac2c7 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:19.614085 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f0ee9-4885-4d6d-9d19-0f115c2c0674 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:19.622161 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:19.673713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63519a-1383-4830-b129-d134e74ac2c7 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:19.680062 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670113e4-c6bb-42f3-b00f-db71a202c951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:19.681195 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670113e4-c6bb-42f3-b00f-db71a202c951 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:19.681682 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670113e4-c6bb-42f3-b00f-db71a202c951 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:19.682172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670113e4-c6bb-42f3-b00f-db71a202c951 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:19.686138 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670113e4-c6bb-42f3-b00f-db71a202c951 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:19.689874 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670113e4-c6bb-42f3-b00f-db71a202c951 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:19.693034 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 167/323] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:37:19.695586 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a9181-1878-45a0-a27f-dfeaabce261f None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:19.695951 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a9181-1878-45a0-a27f-dfeaabce261f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:19.696270 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a9181-1878-45a0-a27f-dfeaabce261f None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:19.698359 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 168/324] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:19.702227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a9181-1878-45a0-a27f-dfeaabce261f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:19.716175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e8462-f5c2-4a39-99f6-c816c5486d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:19.716820 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e8462-f5c2-4a39-99f6-c816c5486d72 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:19.717092 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e8462-f5c2-4a39-99f6-c816c5486d72 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:19.717358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e8462-f5c2-4a39-99f6-c816c5486d72 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:19.731273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117e8462-f5c2-4a39-99f6-c816c5486d72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:19.736117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-117e8462-f5c2-4a39-99f6-c816c5486d72 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:37:19.736117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:19.736117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:19.736117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:19.736117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:19.736117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:19.736117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:19.736117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:19.736117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:19.736117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:19.736117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:19.736117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:19.736117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:19.736117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:19.736117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:19.736117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:19.737170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:19.737170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:19.737170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:19.737170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:19.737170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:19.737170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:19.737170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:19.737170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:19.737170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:19.737170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:19.737170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:19.737170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:19.737170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:19.737170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:19.737170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:19.737170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:19.739323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:19.739323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:19.739323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:19.739323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:19.739323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:19.739323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:19.739323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:19.739323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:19.739323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:19.739323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:19.739323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:37:19.739323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:19.739323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 157/325] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:37:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:37:19.739323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2143e04-5515-4921-a7ae-95166b5db727 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:19.740399 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2143e04-5515-4921-a7ae-95166b5db727 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:19.740399 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2143e04-5515-4921-a7ae-95166b5db727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:19.761230 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-a2143e04-5515-4921-a7ae-95166b5db727 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:19.763440 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a2143e04-5515-4921-a7ae-95166b5db727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:19.857684 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a9181-1878-45a0-a27f-dfeaabce261f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:19.865520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 158/326] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:19.865520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2143e04-5515-4921-a7ae-95166b5db727 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:19.907287 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:19.928950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef60a96-07f5-4c41-bd9a-f0f61080acc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:19.930149 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef60a96-07f5-4c41-bd9a-f0f61080acc0 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:19.930660 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef60a96-07f5-4c41-bd9a-f0f61080acc0 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:19.931174 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef60a96-07f5-4c41-bd9a-f0f61080acc0 None admin] PATH_INFO: `/v3/groups` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:19.932361 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef60a96-07f5-4c41-bd9a-f0f61080acc0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:19.935535 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef60a96-07f5-4c41-bd9a-f0f61080acc0 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:19.955797 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:19.968866 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 169/327] 10.222.0.71 () {60 vars in 1134 bytes} [Fri Aug 26 08:37:19 2022] POST /identity/v3/groups => generated 226 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:37:20.039247 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2143e04-5515-4921-a7ae-95166b5db727 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:20.108380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c616769-27ab-461c-8d4b-80ee825ae31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:20.109414 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c616769-27ab-461c-8d4b-80ee825ae31e None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:20.109870 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c616769-27ab-461c-8d4b-80ee825ae31e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:20.110337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c616769-27ab-461c-8d4b-80ee825ae31e None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:20.111762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c616769-27ab-461c-8d4b-80ee825ae31e None admin] RBAC: Adding query filter params (name=member) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:20.112304 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c616769-27ab-461c-8d4b-80ee825ae31e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:20.115425 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c616769-27ab-461c-8d4b-80ee825ae31e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:20.133734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 159/328] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:37:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:20.143275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef60a96-07f5-4c41-bd9a-f0f61080acc0 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:20.223716 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c616769-27ab-461c-8d4b-80ee825ae31e None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:20.308973 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef60a96-07f5-4c41-bd9a-f0f61080acc0 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:20.386034 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fafc6d-9877-4975-b0fd-b95032647527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:20.387915 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fafc6d-9877-4975-b0fd-b95032647527 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:20.388550 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fafc6d-9877-4975-b0fd-b95032647527 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:20.389069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fafc6d-9877-4975-b0fd-b95032647527 None admin] PATH_INFO: `/v3/users/4aca1ecf02b4498184637dd534f3f157` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:20.390472 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fafc6d-9877-4975-b0fd-b95032647527 None admin] RBAC: Authorizing `identity:get_user(user_id=4aca1ecf02b4498184637dd534f3f157)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:20.392150 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fafc6d-9877-4975-b0fd-b95032647527 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:20.394412 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 170/329] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:37:20 2022] GET /identity/v3/users/4aca1ecf02b4498184637dd534f3f157 => generated 293 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:20.404005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fafc6d-9877-4975-b0fd-b95032647527 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:20.427776 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c616769-27ab-461c-8d4b-80ee825ae31e None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:20.466147 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:20.493953 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:20.509291 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82142931-a36a-408a-a7f6-31df3507b4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:20.511150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82142931-a36a-408a-a7f6-31df3507b4f6 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:20.511630 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82142931-a36a-408a-a7f6-31df3507b4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:20.512069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82142931-a36a-408a-a7f6-31df3507b4f6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:20.531598 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82142931-a36a-408a-a7f6-31df3507b4f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:20.538305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-82142931-a36a-408a-a7f6-31df3507b4f6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 08:37:20.538305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:20.538305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:20.538305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:20.538305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:20.538305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:20.538305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:20.538305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:20.538305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:20.538305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:20.538305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:20.538305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:20.538305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:20.538305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:20.538305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:20.538305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:20.539871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:20.539871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:20.539871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:20.539871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:20.539871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:20.539871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:20.539871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:20.539871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:20.539871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:20.539871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:20.539871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:20.539871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:20.539871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:20.539871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:20.539871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:20.539871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:20.541163 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:20.541163 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:20.541163 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:20.541163 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:20.541163 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:20.541163 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:20.541163 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:20.541163 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:20.541163 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:20.541163 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:20.541163 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 08:37:20.541163 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:20.544314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 160/330] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:37:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:20.547356 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82142931-a36a-408a-a7f6-31df3507b4f6 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:20.603877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fafc6d-9877-4975-b0fd-b95032647527 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:20.676242 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff80b1-c8f4-450d-a023-f5f8d0abaeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:20.677628 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff80b1-c8f4-450d-a023-f5f8d0abaeb6 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:20.678626 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff80b1-c8f4-450d-a023-f5f8d0abaeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:20.679349 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff80b1-c8f4-450d-a023-f5f8d0abaeb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:20.681160 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff80b1-c8f4-450d-a023-f5f8d0abaeb6 None admin] RBAC: Adding query filter params (role.id=b7a48a25b0cd479d85fe5395d64c458e, user.id=4aca1ecf02b4498184637dd534f3f157, scope.system=all) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:20.682026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff80b1-c8f4-450d-a023-f5f8d0abaeb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:20.684857 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff80b1-c8f4-450d-a023-f5f8d0abaeb6 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:20.714579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 171/331] 10.222.0.71 () {56 vars in 1308 bytes} [Fri Aug 26 08:37:20 2022] GET /identity/v3/role_assignments?user.id=4aca1ecf02b4498184637dd534f3f157&scope.system=all&role.id=b7a48a25b0cd479d85fe5395d64c458e => generated 510 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:20.739328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82142931-a36a-408a-a7f6-31df3507b4f6 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:20.793399 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1741b5c-02b0-4718-9991-e52c5475937e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:20.794696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1741b5c-02b0-4718-9991-e52c5475937e None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:20.795393 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1741b5c-02b0-4718-9991-e52c5475937e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:20.795895 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1741b5c-02b0-4718-9991-e52c5475937e None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:20.796848 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1741b5c-02b0-4718-9991-e52c5475937e None admin] RBAC: Adding query filter params (name=reader) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:20.797551 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1741b5c-02b0-4718-9991-e52c5475937e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:20.799744 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1741b5c-02b0-4718-9991-e52c5475937e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:20.814181 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 161/332] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:37:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:20.824041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff80b1-c8f4-450d-a023-f5f8d0abaeb6 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:20.964983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff80b1-c8f4-450d-a023-f5f8d0abaeb6 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:20.984635 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:21.021158 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1de0c0-67b6-4e19-bd38-d416be0e5bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:21.021816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1de0c0-67b6-4e19-bd38-d416be0e5bb4 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:21.022092 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1de0c0-67b6-4e19-bd38-d416be0e5bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:21.022346 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1de0c0-67b6-4e19-bd38-d416be0e5bb4 None admin] PATH_INFO: `/v3/users/afa2909001ab42ab8b35c9f6a894dd15` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:21.026954 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1de0c0-67b6-4e19-bd38-d416be0e5bb4 None admin] RBAC: Authorizing `identity:get_user(user_id=afa2909001ab42ab8b35c9f6a894dd15)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:21.028443 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1de0c0-67b6-4e19-bd38-d416be0e5bb4 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:21.030567 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 172/333] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:37:20 2022] GET /identity/v3/users/afa2909001ab42ab8b35c9f6a894dd15 => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:21.040137 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1741b5c-02b0-4718-9991-e52c5475937e None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:21.105195 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:21.215710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1741b5c-02b0-4718-9991-e52c5475937e None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:21.285299 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccce0d4-263e-4c24-8b30-c6dcbd6af09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:21.286475 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccce0d4-263e-4c24-8b30-c6dcbd6af09a None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:21.287001 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccce0d4-263e-4c24-8b30-c6dcbd6af09a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:21.287478 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccce0d4-263e-4c24-8b30-c6dcbd6af09a None admin] PATH_INFO: `/v3/system/users/afa2909001ab42ab8b35c9f6a894dd15/roles/0f660bd0ff4c4beda97ed7c24851d2d4` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:21.291079 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccce0d4-263e-4c24-8b30-c6dcbd6af09a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=afa2909001ab42ab8b35c9f6a894dd15, role_id=0f660bd0ff4c4beda97ed7c24851d2d4)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:21.293187 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccce0d4-263e-4c24-8b30-c6dcbd6af09a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:21.310279 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 162/334] 10.222.0.71 () {58 vars in 1334 bytes} [Fri Aug 26 08:37:21 2022] PUT /identity/v3/system/users/afa2909001ab42ab8b35c9f6a894dd15/roles/0f660bd0ff4c4beda97ed7c24851d2d4 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:37:23.225964 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8433e6e-76a4-4349-86a3-e84f5d0afc4a None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:23.226794 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8433e6e-76a4-4349-86a3-e84f5d0afc4a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:23.226794 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8433e6e-76a4-4349-86a3-e84f5d0afc4a None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:23.228962 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 173/335] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:23.236810 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2604eb-9577-4c51-8e4b-9e21f5e1cfb0 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:23.237574 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2604eb-9577-4c51-8e4b-9e21f5e1cfb0 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:23.237742 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2604eb-9577-4c51-8e4b-9e21f5e1cfb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:23.270193 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-be2604eb-9577-4c51-8e4b-9e21f5e1cfb0 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:23.271449 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-be2604eb-9577-4c51-8e4b-9e21f5e1cfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:23.385944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 163/336] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:23.483412 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0a5ce-aa65-4013-aac4-49250b6cccd1 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:23.483660 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0a5ce-aa65-4013-aac4-49250b6cccd1 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:23.483970 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0a5ce-aa65-4013-aac4-49250b6cccd1 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:23.485554 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 174/337] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:23.502988 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153e16f-cafa-47e9-a819-2fba90988f39 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:23.503268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153e16f-cafa-47e9-a819-2fba90988f39 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:23.503588 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153e16f-cafa-47e9-a819-2fba90988f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:23.531576 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-0153e16f-cafa-47e9-a819-2fba90988f39 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:23.532889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0153e16f-cafa-47e9-a819-2fba90988f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:23.575963 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:23.641625 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 164/338] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:24.179474 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad76ea-76f3-43ee-9d79-ef4a8a261bdf None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:24.179777 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad76ea-76f3-43ee-9d79-ef4a8a261bdf None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:24.180170 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad76ea-76f3-43ee-9d79-ef4a8a261bdf None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:24.181623 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 175/339] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:24.187443 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a562839-f2ba-4b96-accb-bca2406f0da1 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:24.187783 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a562839-f2ba-4b96-accb-bca2406f0da1 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:24.188049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a562839-f2ba-4b96-accb-bca2406f0da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:24.214938 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-3a562839-f2ba-4b96-accb-bca2406f0da1 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:24.216222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3a562839-f2ba-4b96-accb-bca2406f0da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:24.306201 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad76ea-76f3-43ee-9d79-ef4a8a261bdf None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:24.320690 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 165/340] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 08:37:24.570736 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad76ea-76f3-43ee-9d79-ef4a8a261bdf None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:24.592835 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a562839-f2ba-4b96-accb-bca2406f0da1 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:24.644167 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c82dc-10bc-479e-a481-36d596a7b749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:24.645216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c82dc-10bc-479e-a481-36d596a7b749 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:24.645683 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c82dc-10bc-479e-a481-36d596a7b749 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:24.646123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c82dc-10bc-479e-a481-36d596a7b749 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:24.672457 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0c82dc-10bc-479e-a481-36d596a7b749 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:24.691342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-db0c82dc-10bc-479e-a481-36d596a7b749 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:37:24.691342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:24.691342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:24.691342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:24.691342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:24.691342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:24.691342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:24.691342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:24.691342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:24.691342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:24.691342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:24.691342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:24.691342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:24.691342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:24.691342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:24.691342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:24.693111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:24.693111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:24.693111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:24.693111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:24.693111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:24.693111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:24.693111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:24.693111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:24.693111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:24.693111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:24.693111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:24.693111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:24.693111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:24.693111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:24.693111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:24.693111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:24.694627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:24.694627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:24.694627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:24.694627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:24.694627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:24.694627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:24.694627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:24.694627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:24.694627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:24.694627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:24.694627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:37:24.694627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:24.694627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 176/341] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:37:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 389 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:24.703424 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c82dc-10bc-479e-a481-36d596a7b749 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:24.790610 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:24.795335 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:24.803895 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a562839-f2ba-4b96-accb-bca2406f0da1 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:24.875803 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0d5ea-ab63-4953-bcfc-65ac0b3f7948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:24.876729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0d5ea-ab63-4953-bcfc-65ac0b3f7948 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:24.877197 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0d5ea-ab63-4953-bcfc-65ac0b3f7948 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:24.877617 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d0d5ea-ab63-4953-bcfc-65ac0b3f7948 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:24.892679 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d0d5ea-ab63-4953-bcfc-65ac0b3f7948 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:24.898051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-88d0d5ea-ab63-4953-bcfc-65ac0b3f7948 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 08:37:24.898051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:24.898051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:24.898051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:24.898051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:24.898051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:24.898051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:24.898051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:24.898051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:24.898051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:24.898051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:24.898051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:24.898051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:24.898051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:24.898051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:24.898051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:24.902323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:24.902323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:24.902323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:24.902323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:24.902323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:24.902323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:24.902323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:24.902323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:24.902323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:24.902323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:24.902323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:24.902323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:24.902323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:24.902323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:24.902323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:24.902323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:24.903370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:24.903370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:24.903370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:24.903370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:24.903370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:24.903370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:24.903370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:24.903370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:24.903370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:24.903370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:24.903370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 08:37:24.903370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:24.904170 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c82dc-10bc-479e-a481-36d596a7b749 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:24.906014 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 166/342] 10.222.0.71 () {56 vars in 1114 bytes} [Fri Aug 26 08:37:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:24.907292 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0d5ea-ab63-4953-bcfc-65ac0b3f7948 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:24.996970 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d66153-d557-41b4-8ebe-6ce2309ea2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:24.997994 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d66153-d557-41b4-8ebe-6ce2309ea2e0 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:24.998478 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d66153-d557-41b4-8ebe-6ce2309ea2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:24.998978 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d66153-d557-41b4-8ebe-6ce2309ea2e0 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:25.000188 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d66153-d557-41b4-8ebe-6ce2309ea2e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:25.000725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d66153-d557-41b4-8ebe-6ce2309ea2e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:25.003358 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d66153-d557-41b4-8ebe-6ce2309ea2e0 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:25.038640 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 177/343] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:37:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:25.046652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d66153-d557-41b4-8ebe-6ce2309ea2e0 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:25.198652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d0d5ea-ab63-4953-bcfc-65ac0b3f7948 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:25.282697 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:25.287268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec96703-7774-4233-ab1e-65b98e998623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:25.287991 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d66153-d557-41b4-8ebe-6ce2309ea2e0 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:25.288685 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec96703-7774-4233-ab1e-65b98e998623 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:25.289188 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec96703-7774-4233-ab1e-65b98e998623 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:25.289672 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec96703-7774-4233-ab1e-65b98e998623 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:25.313066 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec96703-7774-4233-ab1e-65b98e998623 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:25.322805 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-9ec96703-7774-4233-ab1e-65b98e998623 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:37:25.322805 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:25.322805 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:25.322805 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:25.322805 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:25.322805 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:25.322805 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:25.322805 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:25.322805 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:25.322805 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:25.322805 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:25.322805 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:25.322805 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:25.322805 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:25.322805 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:25.322805 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:25.324271 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:25.324271 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:25.324271 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:25.324271 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:25.324271 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:25.324271 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:25.324271 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:25.324271 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:25.324271 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:25.324271 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:25.324271 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:25.324271 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:25.324271 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:25.324271 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:25.324271 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:25.324271 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:25.325094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:25.325094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:25.325094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:25.325094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:25.325094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:25.325094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:25.325094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:25.325094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:25.325094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:25.325094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:25.325094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:37:25.325094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:25.325094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 167/344] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:37:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:25.326510 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec96703-7774-4233-ab1e-65b98e998623 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:25.333561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa9320-1491-4383-888e-982ae918d46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:25.335061 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa9320-1491-4383-888e-982ae918d46d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:25.335559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa9320-1491-4383-888e-982ae918d46d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:25.336049 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa9320-1491-4383-888e-982ae918d46d None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:25.336950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa9320-1491-4383-888e-982ae918d46d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:25.337458 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa9320-1491-4383-888e-982ae918d46d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:25.339129 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa9320-1491-4383-888e-982ae918d46d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:25.349201 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 178/345] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:37:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:25.351056 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074c093-84cc-4609-94a0-b617564758c7 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:25.351585 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074c093-84cc-4609-94a0-b617564758c7 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:25.352049 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074c093-84cc-4609-94a0-b617564758c7 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:25.353636 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 179/346] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:25.358669 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:25.360110 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074c093-84cc-4609-94a0-b617564758c7 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:25.503733 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074c093-84cc-4609-94a0-b617564758c7 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:25.520784 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec96703-7774-4233-ab1e-65b98e998623 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:25.565698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef91a5-e9d8-4ccf-87aa-49df948a9116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:25.566691 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef91a5-e9d8-4ccf-87aa-49df948a9116 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:25.567280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef91a5-e9d8-4ccf-87aa-49df948a9116 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:25.567722 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef91a5-e9d8-4ccf-87aa-49df948a9116 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:25.568628 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef91a5-e9d8-4ccf-87aa-49df948a9116 None admin] RBAC: Adding query filter params (name=member) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:25.569120 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef91a5-e9d8-4ccf-87aa-49df948a9116 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:25.572389 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef91a5-e9d8-4ccf-87aa-49df948a9116 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:25.573573 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777ff72-5faf-456b-8797-23fc4c632905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:25.576405 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777ff72-5faf-456b-8797-23fc4c632905 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:25.576882 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777ff72-5faf-456b-8797-23fc4c632905 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:25.577154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777ff72-5faf-456b-8797-23fc4c632905 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:25.584915 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 180/347] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:37:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:25.586673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef91a5-e9d8-4ccf-87aa-49df948a9116 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:25.626311 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:25.680396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777ff72-5faf-456b-8797-23fc4c632905 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:25.682548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777ff72-5faf-456b-8797-23fc4c632905 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:25.701974 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:25.729171 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef91a5-e9d8-4ccf-87aa-49df948a9116 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:25.786567 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1b7cf-058e-413f-8a0a-14e4e9b37de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:25.787184 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1b7cf-058e-413f-8a0a-14e4e9b37de5 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:25.787534 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1b7cf-058e-413f-8a0a-14e4e9b37de5 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:25.787774 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1b7cf-058e-413f-8a0a-14e4e9b37de5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:25.825043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 168/348] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:37:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:37:25.827165 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4deccf8-811c-44e1-9fb4-0be9756134d0 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:25.827457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4deccf8-811c-44e1-9fb4-0be9756134d0 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:25.827828 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4deccf8-811c-44e1-9fb4-0be9756134d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:25.851268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b4deccf8-811c-44e1-9fb4-0be9756134d0 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:25.852510 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b4deccf8-811c-44e1-9fb4-0be9756134d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:25.888947 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1b7cf-058e-413f-8a0a-14e4e9b37de5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:25.891395 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1b7cf-058e-413f-8a0a-14e4e9b37de5 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:25.923039 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:25.950665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 169/349] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:25.960757 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc1c07-0ea5-418e-a20f-6d45602d182f None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:25.961004 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc1c07-0ea5-418e-a20f-6d45602d182f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:25.961281 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc1c07-0ea5-418e-a20f-6d45602d182f None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:25.962703 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 170/350] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:25 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:25.965874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e0da1-18c2-425c-8e17-e503e9f17656 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:25.966361 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e0da1-18c2-425c-8e17-e503e9f17656 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:25.966647 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e0da1-18c2-425c-8e17-e503e9f17656 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:25.968114 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 171/351] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:25.969642 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e0da1-18c2-425c-8e17-e503e9f17656 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:26.043563 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:26.052285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 181/352] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:37:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:37:26.053259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1b7cf-058e-413f-8a0a-14e4e9b37de5 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:26.163750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e0da1-18c2-425c-8e17-e503e9f17656 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:26.217071 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6539e2-b304-42f9-a6fb-dcf9ce865fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:26.218485 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6539e2-b304-42f9-a6fb-dcf9ce865fc2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:26.218485 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6539e2-b304-42f9-a6fb-dcf9ce865fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:26.218485 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6539e2-b304-42f9-a6fb-dcf9ce865fc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:26.236862 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1b7cf-058e-413f-8a0a-14e4e9b37de5 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:26.270393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:26.287853 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff5c36-679d-49b1-9e21-3aa6661de41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:26.288534 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff5c36-679d-49b1-9e21-3aa6661de41b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:26.288814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff5c36-679d-49b1-9e21-3aa6661de41b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:26.289608 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff5c36-679d-49b1-9e21-3aa6661de41b None admin] PATH_INFO: `/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:26.291355 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff5c36-679d-49b1-9e21-3aa6661de41b None admin] RBAC: Authorizing `identity:get_project(project_id=e55f1cc9f9c9481698d6962c7af7fbfe)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:26.292871 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff5c36-679d-49b1-9e21-3aa6661de41b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:26.294632 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 182/353] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:37:26 2022] GET /identity/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:26.295796 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068bc5f-5686-46bf-90cb-906c887e8c04 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:26.296075 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068bc5f-5686-46bf-90cb-906c887e8c04 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:26.296356 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068bc5f-5686-46bf-90cb-906c887e8c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:26.317142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e068bc5f-5686-46bf-90cb-906c887e8c04 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:26.318236 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e068bc5f-5686-46bf-90cb-906c887e8c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:26.318526 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6539e2-b304-42f9-a6fb-dcf9ce865fc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:26.320925 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6539e2-b304-42f9-a6fb-dcf9ce865fc2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:26.388170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:26.427690 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 183/354] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:26.429050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0aa4c4-8eef-4c7e-ba1b-20ccfa23f865 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:26.429340 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0aa4c4-8eef-4c7e-ba1b-20ccfa23f865 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:26.429717 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0aa4c4-8eef-4c7e-ba1b-20ccfa23f865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:26.451922 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-6a0aa4c4-8eef-4c7e-ba1b-20ccfa23f865 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:26.453209 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-6a0aa4c4-8eef-4c7e-ba1b-20ccfa23f865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:26.476264 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 172/355] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:37:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:37:26.477288 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6539e2-b304-42f9-a6fb-dcf9ce865fc2 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:26.548321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 184/356] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:26.550172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0aa4c4-8eef-4c7e-ba1b-20ccfa23f865 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:26.613791 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6539e2-b304-42f9-a6fb-dcf9ce865fc2 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:26.619810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:26.674918 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd517e-e096-4346-bb04-f3c4d5c78ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:26.675587 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd517e-e096-4346-bb04-f3c4d5c78ca4 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:26.675868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd517e-e096-4346-bb04-f3c4d5c78ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:26.676148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd517e-e096-4346-bb04-f3c4d5c78ca4 None admin] PATH_INFO: `/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:26.678184 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd517e-e096-4346-bb04-f3c4d5c78ca4 None admin] RBAC: Authorizing `identity:get_project(project_id=e55f1cc9f9c9481698d6962c7af7fbfe)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:26.680074 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd517e-e096-4346-bb04-f3c4d5c78ca4 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:26.686632 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 173/357] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:37:26 2022] GET /identity/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:26.686632 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd517e-e096-4346-bb04-f3c4d5c78ca4 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:26.701151 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:26.702798 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0aa4c4-8eef-4c7e-ba1b-20ccfa23f865 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:26.751291 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1e95f-c175-4f99-9ba0-2dbf52a45412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:26.751974 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1e95f-c175-4f99-9ba0-2dbf52a45412 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:26.752428 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1e95f-c175-4f99-9ba0-2dbf52a45412 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:26.753382 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1e95f-c175-4f99-9ba0-2dbf52a45412 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:26.771732 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1e95f-c175-4f99-9ba0-2dbf52a45412 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:26.779072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-07e1e95f-c175-4f99-9ba0-2dbf52a45412 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 08:37:26.779072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:26.779072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:26.779072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:26.779072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:26.779072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:26.779072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:26.779072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:26.779072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:26.779072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:26.779072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:26.779072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:26.779072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:26.779072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:26.779072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:26.779072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:26.780840 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:26.780840 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:26.780840 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:26.780840 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:26.780840 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:26.780840 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:26.780840 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:26.780840 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:26.780840 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:26.780840 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:26.780840 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:26.780840 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:26.780840 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:26.780840 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:26.780840 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:26.780840 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:26.782425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:26.782425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:26.782425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:26.782425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:26.782425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:26.782425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:26.782425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:26.782425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:26.782425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:26.782425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:26.782425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 08:37:26.782425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:26.782425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 185/358] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:37:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:26.782425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1e95f-c175-4f99-9ba0-2dbf52a45412 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:26.855546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd517e-e096-4346-bb04-f3c4d5c78ca4 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:26.931167 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5ec24-4437-4417-9ebc-2f4b18be5567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:26.932361 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5ec24-4437-4417-9ebc-2f4b18be5567 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:26.933022 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5ec24-4437-4417-9ebc-2f4b18be5567 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:26.933639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5ec24-4437-4417-9ebc-2f4b18be5567 None admin] PATH_INFO: `/v3/groups/8f8898e63cb545a5b96eb9c08232ba22` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:26.946409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1e95f-c175-4f99-9ba0-2dbf52a45412 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:26.947767 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5ec24-4437-4417-9ebc-2f4b18be5567 None admin] RBAC: Authorizing `identity:get_group(group_id=8f8898e63cb545a5b96eb9c08232ba22)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:26.950029 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5ec24-4437-4417-9ebc-2f4b18be5567 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:26.952564 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 174/359] 10.222.0.71 () {56 vars in 1180 bytes} [Fri Aug 26 08:37:26 2022] GET /identity/v3/groups/8f8898e63cb545a5b96eb9c08232ba22 => generated 226 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:26.953789 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5ec24-4437-4417-9ebc-2f4b18be5567 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:26.973493 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:27.018470 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efb122-a279-489f-b9cf-130ed2bcaa10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:27.019292 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efb122-a279-489f-b9cf-130ed2bcaa10 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:27.019664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efb122-a279-489f-b9cf-130ed2bcaa10 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:27.020004 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98efb122-a279-489f-b9cf-130ed2bcaa10 None admin] PATH_INFO: `/v3/projects/214bc7d9ff5940989811937e08b70389` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:27.022242 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98efb122-a279-489f-b9cf-130ed2bcaa10 None admin] RBAC: Authorizing `identity:get_project(project_id=214bc7d9ff5940989811937e08b70389)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:27.024673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98efb122-a279-489f-b9cf-130ed2bcaa10 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:27.027577 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 186/360] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:37:26 2022] GET /identity/v3/projects/214bc7d9ff5940989811937e08b70389 => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:27.028563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efb122-a279-489f-b9cf-130ed2bcaa10 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:27.086000 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:27.137097 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5ec24-4437-4417-9ebc-2f4b18be5567 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:27.194530 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98efb122-a279-489f-b9cf-130ed2bcaa10 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:27.202627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b54768-d046-462b-8eae-9bedae1b4a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:27.203623 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b54768-d046-462b-8eae-9bedae1b4a44 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:27.204078 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b54768-d046-462b-8eae-9bedae1b4a44 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:27.204520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b54768-d046-462b-8eae-9bedae1b4a44 None admin] PATH_INFO: `/v3/groups/8f8898e63cb545a5b96eb9c08232ba22` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:27.206696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b54768-d046-462b-8eae-9bedae1b4a44 None admin] RBAC: Authorizing `identity:get_group(group_id=8f8898e63cb545a5b96eb9c08232ba22)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:27.209058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b54768-d046-462b-8eae-9bedae1b4a44 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:27.212458 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 175/361] 10.222.0.71 () {56 vars in 1180 bytes} [Fri Aug 26 08:37:26 2022] GET /identity/v3/groups/8f8898e63cb545a5b96eb9c08232ba22 => generated 226 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:27.213957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b54768-d046-462b-8eae-9bedae1b4a44 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:27.260812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19cc1a-d37e-4e87-af26-05b2d1616cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:27.261501 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19cc1a-d37e-4e87-af26-05b2d1616cb1 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:27.261784 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19cc1a-d37e-4e87-af26-05b2d1616cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:27.262060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19cc1a-d37e-4e87-af26-05b2d1616cb1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:27.276589 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c19cc1a-d37e-4e87-af26-05b2d1616cb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:27.283048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-8c19cc1a-d37e-4e87-af26-05b2d1616cb1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 08:37:27.283048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:27.283048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:27.283048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:27.283048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:27.283048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:27.283048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:27.283048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:27.283048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:27.283048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:27.283048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:27.283048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:27.283048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:27.283048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:27.283048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:27.283048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:27.284260 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:27.284260 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:27.284260 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:27.284260 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:27.284260 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:27.284260 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:27.284260 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:27.284260 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:27.284260 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:27.284260 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:27.284260 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:27.284260 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:27.284260 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:27.284260 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:27.284260 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:27.284260 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:27.291977 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:27.291977 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:27.291977 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:27.291977 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:27.291977 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:27.291977 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:27.291977 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:27.291977 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:27.291977 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:27.291977 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:27.291977 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 08:37:27.291977 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:27.291977 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 187/362] 10.222.0.71 () {56 vars in 1114 bytes} [Fri Aug 26 08:37:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:27.291977 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19cc1a-d37e-4e87-af26-05b2d1616cb1 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:27.377339 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:27.418340 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b54768-d046-462b-8eae-9bedae1b4a44 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:27.467858 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90547198-7657-48da-878c-64d86bdd2dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:27.468663 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90547198-7657-48da-878c-64d86bdd2dc7 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:27.468975 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90547198-7657-48da-878c-64d86bdd2dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:27.469313 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90547198-7657-48da-878c-64d86bdd2dc7 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:27.470236 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90547198-7657-48da-878c-64d86bdd2dc7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:27.470603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90547198-7657-48da-878c-64d86bdd2dc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:27.472384 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90547198-7657-48da-878c-64d86bdd2dc7 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:27.476274 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19cc1a-d37e-4e87-af26-05b2d1616cb1 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:27.483545 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 176/363] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:37:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:27.483747 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90547198-7657-48da-878c-64d86bdd2dc7 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:27.493805 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:27.520916 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c1f3c-5cfd-4067-bdb3-5586b28e36e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:27.521675 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c1f3c-5cfd-4067-bdb3-5586b28e36e6 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:27.521997 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c1f3c-5cfd-4067-bdb3-5586b28e36e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:27.522314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c1f3c-5cfd-4067-bdb3-5586b28e36e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:27.523154 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112c1f3c-5cfd-4067-bdb3-5586b28e36e6 None admin] RBAC: Adding query filter params (group.id=8f8898e63cb545a5b96eb9c08232ba22, role.id=b7a48a25b0cd479d85fe5395d64c458e, scope.project.id=e55f1cc9f9c9481698d6962c7af7fbfe) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:27.523441 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112c1f3c-5cfd-4067-bdb3-5586b28e36e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:27.525124 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112c1f3c-5cfd-4067-bdb3-5586b28e36e6 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:27.531493 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 188/364] 10.222.0.71 () {56 vars in 1376 bytes} [Fri Aug 26 08:37:27 2022] GET /identity/v3/role_assignments?group.id=8f8898e63cb545a5b96eb9c08232ba22&scope.project.id=e55f1cc9f9c9481698d6962c7af7fbfe&role.id=b7a48a25b0cd479d85fe5395d64c458e => generated 260 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:27.532507 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c1f3c-5cfd-4067-bdb3-5586b28e36e6 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:27.636358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90547198-7657-48da-878c-64d86bdd2dc7 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:27.673278 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c1f3c-5cfd-4067-bdb3-5586b28e36e6 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:27.676243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:27.679275 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbaab6f-6a1d-4526-9fd1-d068a470c515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:27.679930 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbaab6f-6a1d-4526-9fd1-d068a470c515 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:27.680203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbaab6f-6a1d-4526-9fd1-d068a470c515 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:27.680476 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbaab6f-6a1d-4526-9fd1-d068a470c515 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:27.692747 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbaab6f-6a1d-4526-9fd1-d068a470c515 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:27.698334 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-3bbaab6f-6a1d-4526-9fd1-d068a470c515 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:37:27.698334 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:27.698334 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:27.698334 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:27.698334 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:27.698334 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:27.698334 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:27.698334 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:27.698334 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:27.698334 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:27.698334 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:27.698334 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:27.698334 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:27.698334 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:27.698334 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:27.698334 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:27.699957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:27.699957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:27.699957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:27.699957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:27.699957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:27.699957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:27.699957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:27.699957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:27.699957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:27.699957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:27.699957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:27.699957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:27.699957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:27.699957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:27.699957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:27.699957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:27.702273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:27.702273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:27.702273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:27.702273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:27.702273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:27.702273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:27.702273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:27.702273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:27.702273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:27.702273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:27.702273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:37:27.702273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:27.702273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 177/365] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:37:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:27.702273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbaab6f-6a1d-4526-9fd1-d068a470c515 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:27.726645 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4571512-73dd-4b1c-9d51-db0a5491d692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:27.727661 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4571512-73dd-4b1c-9d51-db0a5491d692 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:27.728073 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4571512-73dd-4b1c-9d51-db0a5491d692 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:27.728401 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4571512-73dd-4b1c-9d51-db0a5491d692 None admin] PATH_INFO: `/v3/groups/8f8898e63cb545a5b96eb9c08232ba22` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:27.729693 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4571512-73dd-4b1c-9d51-db0a5491d692 None admin] RBAC: Authorizing `identity:get_group(group_id=8f8898e63cb545a5b96eb9c08232ba22)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:27.731307 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4571512-73dd-4b1c-9d51-db0a5491d692 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:27.737726 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 189/366] 10.222.0.71 () {56 vars in 1180 bytes} [Fri Aug 26 08:37:27 2022] GET /identity/v3/groups/8f8898e63cb545a5b96eb9c08232ba22 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:27.737726 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4571512-73dd-4b1c-9d51-db0a5491d692 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:27.796667 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:27.843411 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbaab6f-6a1d-4526-9fd1-d068a470c515 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:27.857711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4571512-73dd-4b1c-9d51-db0a5491d692 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:27.886011 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd764d20-5e03-4584-8a53-3ffaae3313b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:27.887007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd764d20-5e03-4584-8a53-3ffaae3313b2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:27.887436 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd764d20-5e03-4584-8a53-3ffaae3313b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:27.887818 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd764d20-5e03-4584-8a53-3ffaae3313b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:27.888702 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd764d20-5e03-4584-8a53-3ffaae3313b2 None admin] RBAC: Adding query filter params (group.id=8f8898e63cb545a5b96eb9c08232ba22, role.id=6195dc21ae4749daa31797d2c42e050e, scope.project.id=e55f1cc9f9c9481698d6962c7af7fbfe) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:27.889018 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd764d20-5e03-4584-8a53-3ffaae3313b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:27.890495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd764d20-5e03-4584-8a53-3ffaae3313b2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:27.906930 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 178/367] 10.222.0.71 () {56 vars in 1376 bytes} [Fri Aug 26 08:37:27 2022] GET /identity/v3/role_assignments?group.id=8f8898e63cb545a5b96eb9c08232ba22&scope.project.id=e55f1cc9f9c9481698d6962c7af7fbfe&role.id=6195dc21ae4749daa31797d2c42e050e => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:27.906930 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd764d20-5e03-4584-8a53-3ffaae3313b2 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:27.912044 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265a929-f0a5-45c2-a971-2019950a335c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:27.912785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265a929-f0a5-45c2-a971-2019950a335c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:27.913130 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265a929-f0a5-45c2-a971-2019950a335c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:27.913470 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8265a929-f0a5-45c2-a971-2019950a335c None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:27.914241 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8265a929-f0a5-45c2-a971-2019950a335c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:27.914615 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8265a929-f0a5-45c2-a971-2019950a335c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:27.916433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8265a929-f0a5-45c2-a971-2019950a335c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:27.927512 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 190/368] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:37:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:27.928710 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265a929-f0a5-45c2-a971-2019950a335c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:27.971207 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:28.057142 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd764d20-5e03-4584-8a53-3ffaae3313b2 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:28.065292 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8265a929-f0a5-45c2-a971-2019950a335c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:28.094347 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:28.107315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b339c253-d749-4dde-b3be-3b57bce6c9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:28.107986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339c253-d749-4dde-b3be-3b57bce6c9b4 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:28.108378 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339c253-d749-4dde-b3be-3b57bce6c9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:28.108800 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339c253-d749-4dde-b3be-3b57bce6c9b4 None admin] PATH_INFO: `/v3/users/afa2909001ab42ab8b35c9f6a894dd15` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:28.110303 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b339c253-d749-4dde-b3be-3b57bce6c9b4 None admin] RBAC: Authorizing `identity:get_user(user_id=afa2909001ab42ab8b35c9f6a894dd15)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:28.111870 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b339c253-d749-4dde-b3be-3b57bce6c9b4 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:28.119144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 179/369] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:37:27 2022] GET /identity/v3/users/afa2909001ab42ab8b35c9f6a894dd15 => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:28.119144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b339c253-d749-4dde-b3be-3b57bce6c9b4 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:28.120546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3984bd2-5c16-498e-b749-ba6396d1320a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:28.120692 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3984bd2-5c16-498e-b749-ba6396d1320a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:28.120908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3984bd2-5c16-498e-b749-ba6396d1320a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:28.121414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3984bd2-5c16-498e-b749-ba6396d1320a None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:28.121870 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3984bd2-5c16-498e-b749-ba6396d1320a None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:28.122175 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3984bd2-5c16-498e-b749-ba6396d1320a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:28.123665 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3984bd2-5c16-498e-b749-ba6396d1320a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:28.134168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 191/370] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:37:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:28.134833 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3984bd2-5c16-498e-b749-ba6396d1320a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:28.268595 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:28.281545 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3984bd2-5c16-498e-b749-ba6396d1320a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:28.282978 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b339c253-d749-4dde-b3be-3b57bce6c9b4 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:28.332064 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7cf051-f246-4009-a717-3a3a3ce85b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:28.332911 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7cf051-f246-4009-a717-3a3a3ce85b0a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:28.333257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7cf051-f246-4009-a717-3a3a3ce85b0a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:28.333531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7cf051-f246-4009-a717-3a3a3ce85b0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:28.334279 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7cf051-f246-4009-a717-3a3a3ce85b0a None admin] RBAC: Adding query filter params (group.id=8f8898e63cb545a5b96eb9c08232ba22, role.id=b7a48a25b0cd479d85fe5395d64c458e, scope.project.id=214bc7d9ff5940989811937e08b70389) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:28.334594 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7cf051-f246-4009-a717-3a3a3ce85b0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:28.336295 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7cf051-f246-4009-a717-3a3a3ce85b0a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:28.342559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bad85c-032e-431d-b396-bf94982009ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:28.343328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 180/371] 10.222.0.71 () {56 vars in 1376 bytes} [Fri Aug 26 08:37:28 2022] GET /identity/v3/role_assignments?group.id=8f8898e63cb545a5b96eb9c08232ba22&scope.project.id=214bc7d9ff5940989811937e08b70389&role.id=b7a48a25b0cd479d85fe5395d64c458e => generated 260 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:28.343949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bad85c-032e-431d-b396-bf94982009ee None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:28.344177 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7cf051-f246-4009-a717-3a3a3ce85b0a None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:28.344747 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bad85c-032e-431d-b396-bf94982009ee None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:28.345216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bad85c-032e-431d-b396-bf94982009ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:28.418546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:28.441704 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bad85c-032e-431d-b396-bf94982009ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:28.444127 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bad85c-032e-431d-b396-bf94982009ee None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:28.468776 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7cf051-f246-4009-a717-3a3a3ce85b0a None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:28.509852 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56abf245-177d-4c6d-88e1-8bc973ea9d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:28.510488 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abf245-177d-4c6d-88e1-8bc973ea9d33 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:28.510769 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abf245-177d-4c6d-88e1-8bc973ea9d33 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:28.511309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abf245-177d-4c6d-88e1-8bc973ea9d33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:28.511982 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56abf245-177d-4c6d-88e1-8bc973ea9d33 None admin] RBAC: Adding query filter params (role.id=0f660bd0ff4c4beda97ed7c24851d2d4, user.id=afa2909001ab42ab8b35c9f6a894dd15, scope.system=all) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:28.512305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56abf245-177d-4c6d-88e1-8bc973ea9d33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:28.514085 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56abf245-177d-4c6d-88e1-8bc973ea9d33 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:28.535051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 181/372] 10.222.0.71 () {56 vars in 1308 bytes} [Fri Aug 26 08:37:28 2022] GET /identity/v3/role_assignments?user.id=afa2909001ab42ab8b35c9f6a894dd15&scope.system=all&role.id=0f660bd0ff4c4beda97ed7c24851d2d4 => generated 510 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:28.535051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56abf245-177d-4c6d-88e1-8bc973ea9d33 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:28.611920 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:28.625591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 192/373] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:37:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:37:28.634506 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bad85c-032e-431d-b396-bf94982009ee None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:28.768985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56abf245-177d-4c6d-88e1-8bc973ea9d33 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:28.799385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:28.819705 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebe72b8-c0b5-4ceb-b1d9-8e7b6218cd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:28.820363 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe72b8-c0b5-4ceb-b1d9-8e7b6218cd52 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:28.820784 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe72b8-c0b5-4ceb-b1d9-8e7b6218cd52 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:28.821065 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe72b8-c0b5-4ceb-b1d9-8e7b6218cd52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:28.850094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bad85c-032e-431d-b396-bf94982009ee None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:28.914432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435a686-fa15-4de4-a877-07b95b5118ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:28.915555 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435a686-fa15-4de4-a877-07b95b5118ad None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:28.915846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435a686-fa15-4de4-a877-07b95b5118ad None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:28.916219 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435a686-fa15-4de4-a877-07b95b5118ad None admin] PATH_INFO: `/v3/projects/214bc7d9ff5940989811937e08b70389` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:28.925211 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f435a686-fa15-4de4-a877-07b95b5118ad None admin] RBAC: Authorizing `identity:get_project(project_id=214bc7d9ff5940989811937e08b70389)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:28.927200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f435a686-fa15-4de4-a877-07b95b5118ad None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:28.933437 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 193/374] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:37:28 2022] GET /identity/v3/projects/214bc7d9ff5940989811937e08b70389 => generated 302 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:28.939336 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435a686-fa15-4de4-a877-07b95b5118ad None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:28.989848 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebe72b8-c0b5-4ceb-b1d9-8e7b6218cd52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:28.991786 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebe72b8-c0b5-4ceb-b1d9-8e7b6218cd52 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:29.088349 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:29.117789 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 182/375] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:37:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:37:29.123516 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebe72b8-c0b5-4ceb-b1d9-8e7b6218cd52 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:29.144012 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:29.201824 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435a686-fa15-4de4-a877-07b95b5118ad None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:29.235656 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebe72b8-c0b5-4ceb-b1d9-8e7b6218cd52 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:29.272903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38126d34-1444-49a6-9d8b-a8f1550dfa43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:29.273630 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38126d34-1444-49a6-9d8b-a8f1550dfa43 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:29.274023 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38126d34-1444-49a6-9d8b-a8f1550dfa43 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:29.274299 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38126d34-1444-49a6-9d8b-a8f1550dfa43 None admin] PATH_INFO: `/v3/projects/45132fb08c454ebaa1180c0b933c68b7` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:29.275688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38126d34-1444-49a6-9d8b-a8f1550dfa43 None admin] RBAC: Authorizing `identity:get_project(project_id=45132fb08c454ebaa1180c0b933c68b7)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:29.277172 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38126d34-1444-49a6-9d8b-a8f1550dfa43 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:29.279487 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 183/376] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:37:29 2022] GET /identity/v3/projects/45132fb08c454ebaa1180c0b933c68b7 => generated 344 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:29.285482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38126d34-1444-49a6-9d8b-a8f1550dfa43 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:29.299767 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295bf639-c993-4d1f-aed7-8ec7c4b0b866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:29.300603 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295bf639-c993-4d1f-aed7-8ec7c4b0b866 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:29.300950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295bf639-c993-4d1f-aed7-8ec7c4b0b866 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:29.301333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295bf639-c993-4d1f-aed7-8ec7c4b0b866 None admin] PATH_INFO: `/v3/groups/8f8898e63cb545a5b96eb9c08232ba22` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:29.304725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295bf639-c993-4d1f-aed7-8ec7c4b0b866 None admin] RBAC: Authorizing `identity:get_group(group_id=8f8898e63cb545a5b96eb9c08232ba22)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:29.306315 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295bf639-c993-4d1f-aed7-8ec7c4b0b866 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:29.308508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 194/377] 10.222.0.71 () {56 vars in 1180 bytes} [Fri Aug 26 08:37:28 2022] GET /identity/v3/groups/8f8898e63cb545a5b96eb9c08232ba22 => generated 226 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:37:29.314704 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295bf639-c993-4d1f-aed7-8ec7c4b0b866 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:29.387338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:29.407694 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38126d34-1444-49a6-9d8b-a8f1550dfa43 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:29.491333 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad0d422-7a6f-4ea2-bbb0-b25ae2be7f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:29.492119 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad0d422-7a6f-4ea2-bbb0-b25ae2be7f9f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:29.492374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad0d422-7a6f-4ea2-bbb0-b25ae2be7f9f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:29.492637 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad0d422-7a6f-4ea2-bbb0-b25ae2be7f9f None admin] PATH_INFO: `/v3/groups/0380bf8970ea4d918c5197fb86dc6412` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:29.511261 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad0d422-7a6f-4ea2-bbb0-b25ae2be7f9f None admin] RBAC: Authorizing `identity:get_group(group_id=0380bf8970ea4d918c5197fb86dc6412)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:29.513742 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad0d422-7a6f-4ea2-bbb0-b25ae2be7f9f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:29.515529 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 184/378] 10.222.0.71 () {56 vars in 1180 bytes} [Fri Aug 26 08:37:29 2022] GET /identity/v3/groups/0380bf8970ea4d918c5197fb86dc6412 => generated 229 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:29.524207 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad0d422-7a6f-4ea2-bbb0-b25ae2be7f9f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:29.568207 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295bf639-c993-4d1f-aed7-8ec7c4b0b866 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:29.614050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:29.631815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf8887-ecde-4619-8b41-b3ef7eca5941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:29.632795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf8887-ecde-4619-8b41-b3ef7eca5941 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:29.633082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf8887-ecde-4619-8b41-b3ef7eca5941 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:29.633354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf8887-ecde-4619-8b41-b3ef7eca5941 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:29.634137 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf8887-ecde-4619-8b41-b3ef7eca5941 None admin] RBAC: Adding query filter params (group.id=8f8898e63cb545a5b96eb9c08232ba22, role.id=6195dc21ae4749daa31797d2c42e050e, scope.project.id=214bc7d9ff5940989811937e08b70389) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:29.634660 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf8887-ecde-4619-8b41-b3ef7eca5941 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:29.638819 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf8887-ecde-4619-8b41-b3ef7eca5941 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:29.652207 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 195/379] 10.222.0.71 () {56 vars in 1376 bytes} [Fri Aug 26 08:37:29 2022] GET /identity/v3/role_assignments?group.id=8f8898e63cb545a5b96eb9c08232ba22&scope.project.id=214bc7d9ff5940989811937e08b70389&role.id=6195dc21ae4749daa31797d2c42e050e => generated 260 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:37:29.690132 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad0d422-7a6f-4ea2-bbb0-b25ae2be7f9f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:29.765567 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb36b5-aee7-4982-9292-7c29c34b91e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:29.766504 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb36b5-aee7-4982-9292-7c29c34b91e6 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:29.767056 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb36b5-aee7-4982-9292-7c29c34b91e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:29.767566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb36b5-aee7-4982-9292-7c29c34b91e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:29.769271 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb36b5-aee7-4982-9292-7c29c34b91e6 None admin] RBAC: Adding query filter params (group.id=0380bf8970ea4d918c5197fb86dc6412, role.id=18c3c8a821b647f7982a955880bd0a9a, scope.project.id=45132fb08c454ebaa1180c0b933c68b7) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:29.769776 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb36b5-aee7-4982-9292-7c29c34b91e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:29.772414 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb36b5-aee7-4982-9292-7c29c34b91e6 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:29.786524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 185/380] 10.222.0.71 () {56 vars in 1376 bytes} [Fri Aug 26 08:37:29 2022] GET /identity/v3/role_assignments?group.id=0380bf8970ea4d918c5197fb86dc6412&scope.project.id=45132fb08c454ebaa1180c0b933c68b7&role.id=18c3c8a821b647f7982a955880bd0a9a => generated 260 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:30.043955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cdbd7-8fc9-4ff5-b384-499264132a5d None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:30.045010 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cdbd7-8fc9-4ff5-b384-499264132a5d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:30.045786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cdbd7-8fc9-4ff5-b384-499264132a5d None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:30.048545 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 196/381] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:30.055404 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83128372-5ec0-4a07-9d35-6524c12bca51 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:30.056303 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83128372-5ec0-4a07-9d35-6524c12bca51 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:30.057010 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83128372-5ec0-4a07-9d35-6524c12bca51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:30.095594 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-83128372-5ec0-4a07-9d35-6524c12bca51 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:30.098304 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-83128372-5ec0-4a07-9d35-6524c12bca51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:30.123666 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:30.205122 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 186/382] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:30.383609 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3ce3c-c5f6-464f-9eff-d6873768be7f None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:30.384297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3ce3c-c5f6-464f-9eff-d6873768be7f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:30.384773 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3ce3c-c5f6-464f-9eff-d6873768be7f None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:30.386788 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 197/383] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:30.392404 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709fcea-2708-4d47-8c8b-5feee07add35 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:30.393096 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709fcea-2708-4d47-8c8b-5feee07add35 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:30.393668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709fcea-2708-4d47-8c8b-5feee07add35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:30.419359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-7709fcea-2708-4d47-8c8b-5feee07add35 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:30.420928 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7709fcea-2708-4d47-8c8b-5feee07add35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:30.518998 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 187/384] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:30.573789 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3ce3c-c5f6-464f-9eff-d6873768be7f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:30.753020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3ce3c-c5f6-464f-9eff-d6873768be7f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:30.809037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1838a-d876-4172-8cce-801f89187519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:30.809717 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1838a-d876-4172-8cce-801f89187519 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:30.809991 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1838a-d876-4172-8cce-801f89187519 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:30.810255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1838a-d876-4172-8cce-801f89187519 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:30.821899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1838a-d876-4172-8cce-801f89187519 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:30.827338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-50f1838a-d876-4172-8cce-801f89187519 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:37:30.827338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:30.827338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:30.827338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:30.827338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:30.827338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:30.827338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:30.827338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:30.827338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:30.827338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:30.827338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:30.827338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:30.827338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:30.827338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:30.827338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:30.827338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:30.828261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:30.828261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:30.828261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:30.828261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:30.828261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:30.828261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:30.828261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:30.828261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:30.828261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:30.828261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:30.828261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:30.828261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:30.828261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:30.828261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:30.828261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:30.828261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:30.829068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:30.829068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:30.829068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:30.829068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:30.829068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:30.829068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:30.829068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:30.829068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:30.829068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:30.829068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:30.829068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:37:30.829068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:30.829669 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 198/385] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:37:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:30.836785 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709fcea-2708-4d47-8c8b-5feee07add35 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:30.872497 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1838a-d876-4172-8cce-801f89187519 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:30.958111 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709fcea-2708-4d47-8c8b-5feee07add35 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:30.967488 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:30.994834 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:30.999187 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf2696-8e5d-43ae-adb2-d6ea742db123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:31.000937 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf2696-8e5d-43ae-adb2-d6ea742db123 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:31.001458 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf2696-8e5d-43ae-adb2-d6ea742db123 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:31.001927 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf2696-8e5d-43ae-adb2-d6ea742db123 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:31.002987 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf2696-8e5d-43ae-adb2-d6ea742db123 None admin] RBAC: Adding query filter params (name=member) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:31.003545 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf2696-8e5d-43ae-adb2-d6ea742db123 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:31.005126 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf2696-8e5d-43ae-adb2-d6ea742db123 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:31.017694 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 188/386] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:37:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:31.020614 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf2696-8e5d-43ae-adb2-d6ea742db123 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:31.102196 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1838a-d876-4172-8cce-801f89187519 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:31.159626 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358c7b8-ea07-45db-afd5-43a9585e9d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:31.160655 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358c7b8-ea07-45db-afd5-43a9585e9d5f None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:31.161153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358c7b8-ea07-45db-afd5-43a9585e9d5f None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:31.161616 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358c7b8-ea07-45db-afd5-43a9585e9d5f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:31.186857 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358c7b8-ea07-45db-afd5-43a9585e9d5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:31.192664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-1358c7b8-ea07-45db-afd5-43a9585e9d5f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 08:37:31.192664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:31.192664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:31.192664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:31.192664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:31.192664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:31.192664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:31.192664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:31.192664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:31.192664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:31.192664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:31.192664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:31.192664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:31.192664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:31.192664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:31.192664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:31.193591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:31.193591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:31.193591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:31.193591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:31.193591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:31.193591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:31.193591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:31.193591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:31.193591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:31.193591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:31.193591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:31.193591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:31.193591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:31.193591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:31.193591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:31.193591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:31.194380 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:31.194380 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:31.194380 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:31.194380 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:31.194380 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:31.194380 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:31.194380 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:31.194380 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:31.194380 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:31.194380 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:31.194380 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 08:37:31.194380 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:31.197561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 199/387] 10.222.0.71 () {56 vars in 1114 bytes} [Fri Aug 26 08:37:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:31.203809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358c7b8-ea07-45db-afd5-43a9585e9d5f None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:31.243363 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf2696-8e5d-43ae-adb2-d6ea742db123 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:31.284053 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244640b-92e3-4f16-996e-18c31e46ef02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:31.284924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244640b-92e3-4f16-996e-18c31e46ef02 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:31.285365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244640b-92e3-4f16-996e-18c31e46ef02 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:31.285774 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244640b-92e3-4f16-996e-18c31e46ef02 None admin] PATH_INFO: `/v3/groups/8f8898e63cb545a5b96eb9c08232ba22` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:31.287671 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244640b-92e3-4f16-996e-18c31e46ef02 None admin] RBAC: Authorizing `identity:get_group(group_id=8f8898e63cb545a5b96eb9c08232ba22)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:31.290159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244640b-92e3-4f16-996e-18c31e46ef02 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:31.292736 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 189/388] 10.222.0.71 () {56 vars in 1180 bytes} [Fri Aug 26 08:37:31 2022] GET /identity/v3/groups/8f8898e63cb545a5b96eb9c08232ba22 => generated 226 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:31.298801 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f09d5-22cc-4502-acc3-4482127604fd None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:31.298801 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f09d5-22cc-4502-acc3-4482127604fd None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:31.298801 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f09d5-22cc-4502-acc3-4482127604fd None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:31.298801 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 190/389] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:31.302260 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f09d5-22cc-4502-acc3-4482127604fd None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:31.323280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:31.381160 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358c7b8-ea07-45db-afd5-43a9585e9d5f None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:31.401633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:31.421321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ccba3f-8bec-44b1-82a3-48ce0e65a52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:31.422077 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ccba3f-8bec-44b1-82a3-48ce0e65a52e None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:31.422453 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ccba3f-8bec-44b1-82a3-48ce0e65a52e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:31.422816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ccba3f-8bec-44b1-82a3-48ce0e65a52e None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:31.423629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ccba3f-8bec-44b1-82a3-48ce0e65a52e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:31.424026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ccba3f-8bec-44b1-82a3-48ce0e65a52e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:31.425578 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ccba3f-8bec-44b1-82a3-48ce0e65a52e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:31.439054 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 200/390] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:37:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:31.440791 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e309a5e-cc87-4e51-9f94-e4ecc7b6a9a0 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:31.441071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e309a5e-cc87-4e51-9f94-e4ecc7b6a9a0 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:31.441351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e309a5e-cc87-4e51-9f94-e4ecc7b6a9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:31.445641 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f09d5-22cc-4502-acc3-4482127604fd None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:31.463459 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-2e309a5e-cc87-4e51-9f94-e4ecc7b6a9a0 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:31.465036 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2e309a5e-cc87-4e51-9f94-e4ecc7b6a9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:31.482268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a055a-8ec3-4621-82eb-3452ee89b7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:31.482934 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a055a-8ec3-4621-82eb-3452ee89b7c3 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:31.483214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a055a-8ec3-4621-82eb-3452ee89b7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:31.483486 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a055a-8ec3-4621-82eb-3452ee89b7c3 None admin] PATH_INFO: `/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:31.484656 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a055a-8ec3-4621-82eb-3452ee89b7c3 None admin] RBAC: Authorizing `identity:get_project(project_id=e55f1cc9f9c9481698d6962c7af7fbfe)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:31.486261 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a055a-8ec3-4621-82eb-3452ee89b7c3 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:31.493532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 191/391] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:37:31 2022] GET /identity/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:31.493532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a055a-8ec3-4621-82eb-3452ee89b7c3 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:31.548754 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 201/392] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:31.549917 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e309a5e-cc87-4e51-9f94-e4ecc7b6a9a0 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:31.582806 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:31.620328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a055a-8ec3-4621-82eb-3452ee89b7c3 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:31.659165 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc161bdc-a0ae-4943-ba61-eeb04fbc0d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:31.660287 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc161bdc-a0ae-4943-ba61-eeb04fbc0d67 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:31.660888 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc161bdc-a0ae-4943-ba61-eeb04fbc0d67 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:31.661344 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc161bdc-a0ae-4943-ba61-eeb04fbc0d67 None admin] PATH_INFO: `/v3/groups/8f8898e63cb545a5b96eb9c08232ba22` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:31.662743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc161bdc-a0ae-4943-ba61-eeb04fbc0d67 None admin] RBAC: Authorizing `identity:get_group(group_id=8f8898e63cb545a5b96eb9c08232ba22)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:31.664525 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc161bdc-a0ae-4943-ba61-eeb04fbc0d67 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:31.666397 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 192/393] 10.222.0.71 () {56 vars in 1180 bytes} [Fri Aug 26 08:37:31 2022] GET /identity/v3/groups/8f8898e63cb545a5b96eb9c08232ba22 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:31.671873 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc161bdc-a0ae-4943-ba61-eeb04fbc0d67 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:31.733095 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e309a5e-cc87-4e51-9f94-e4ecc7b6a9a0 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:31.757124 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:31.824672 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc161bdc-a0ae-4943-ba61-eeb04fbc0d67 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:31.839862 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a4ed7-fe4d-40de-afe7-73fee970f2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:31.840612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a4ed7-fe4d-40de-afe7-73fee970f2a1 None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:31.841010 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a4ed7-fe4d-40de-afe7-73fee970f2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:31.841422 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a4ed7-fe4d-40de-afe7-73fee970f2a1 None admin] PATH_INFO: `/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe/groups/8f8898e63cb545a5b96eb9c08232ba22/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:31.848635 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a4ed7-fe4d-40de-afe7-73fee970f2a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=e55f1cc9f9c9481698d6962c7af7fbfe, group_id=8f8898e63cb545a5b96eb9c08232ba22, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:31.850547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a4ed7-fe4d-40de-afe7-73fee970f2a1 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:31.879671 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 202/394] 10.222.0.71 () {58 vars in 1442 bytes} [Fri Aug 26 08:37:31 2022] PUT /identity/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe/groups/8f8898e63cb545a5b96eb9c08232ba22/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:37:31.880498 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b80031-a8fa-49c3-abcf-489e28679f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:31.881525 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b80031-a8fa-49c3-abcf-489e28679f8c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:31.882034 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b80031-a8fa-49c3-abcf-489e28679f8c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:31.882551 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b80031-a8fa-49c3-abcf-489e28679f8c None admin] PATH_INFO: `/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:31.884433 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b80031-a8fa-49c3-abcf-489e28679f8c None admin] RBAC: Authorizing `identity:get_project(project_id=e55f1cc9f9c9481698d6962c7af7fbfe)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:31.886687 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b80031-a8fa-49c3-abcf-489e28679f8c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:31.889834 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 193/395] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:37:31 2022] GET /identity/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:37:31.896302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a4ed7-fe4d-40de-afe7-73fee970f2a1 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:32.029544 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b80031-a8fa-49c3-abcf-489e28679f8c None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:32.059997 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:32.089631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a4ed7-fe4d-40de-afe7-73fee970f2a1 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:32.137884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5e09f-b96c-4074-b35d-2848ff22e3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:32.138566 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5e09f-b96c-4074-b35d-2848ff22e3d0 None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:32.138840 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5e09f-b96c-4074-b35d-2848ff22e3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:32.139154 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5e09f-b96c-4074-b35d-2848ff22e3d0 None admin] PATH_INFO: `/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe/groups/8f8898e63cb545a5b96eb9c08232ba22/roles/6195dc21ae4749daa31797d2c42e050e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:32.141446 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f5e09f-b96c-4074-b35d-2848ff22e3d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=e55f1cc9f9c9481698d6962c7af7fbfe, group_id=8f8898e63cb545a5b96eb9c08232ba22, role_id=6195dc21ae4749daa31797d2c42e050e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:32.143201 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f5e09f-b96c-4074-b35d-2848ff22e3d0 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:32.164602 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 203/396] 10.222.0.71 () {58 vars in 1442 bytes} [Fri Aug 26 08:37:31 2022] PUT /identity/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe/groups/8f8898e63cb545a5b96eb9c08232ba22/roles/6195dc21ae4749daa31797d2c42e050e => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:37:32.169177 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a73de3-5566-4055-a411-b898c392dd18 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:32.169442 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a73de3-5566-4055-a411-b898c392dd18 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:32.169725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a73de3-5566-4055-a411-b898c392dd18 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:32.172275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 204/397] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:32.178838 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27e0bd-fe1a-474e-925b-d6b6f346246a None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:32.179197 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27e0bd-fe1a-474e-925b-d6b6f346246a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:32.179430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27e0bd-fe1a-474e-925b-d6b6f346246a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:32.201440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-5f27e0bd-fe1a-474e-925b-d6b6f346246a None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:32.202657 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-5f27e0bd-fe1a-474e-925b-d6b6f346246a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:32.210926 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b80031-a8fa-49c3-abcf-489e28679f8c None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:32.233136 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:32.265309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d30488-172f-4427-8aa6-25b7388cff84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:32.266032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d30488-172f-4427-8aa6-25b7388cff84 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:32.266317 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d30488-172f-4427-8aa6-25b7388cff84 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:32.266593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d30488-172f-4427-8aa6-25b7388cff84 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:32.282295 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d30488-172f-4427-8aa6-25b7388cff84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:32.289985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-f6d30488-172f-4427-8aa6-25b7388cff84 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:37:32.289985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:32.289985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:32.289985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:32.289985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:32.289985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:32.289985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:32.289985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:32.289985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:32.289985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:32.289985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:32.289985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:32.289985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:32.289985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:32.289985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:32.289985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:32.290978 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:32.290978 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:32.290978 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:32.290978 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:32.290978 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:32.290978 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:32.290978 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:32.290978 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:32.290978 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:32.290978 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:32.290978 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:32.290978 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:32.290978 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:32.290978 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:32.290978 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:32.290978 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:32.291769 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:32.291769 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:32.291769 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:32.291769 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:32.291769 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:32.291769 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:32.291769 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:32.291769 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:32.291769 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:32.291769 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:32.291769 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:37:32.291769 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:32.292946 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 194/398] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:37:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:32.299575 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d30488-172f-4427-8aa6-25b7388cff84 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:32.340831 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 205/399] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:32.445252 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:32.485258 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d30488-172f-4427-8aa6-25b7388cff84 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:32.524583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4525ac-2644-4168-8576-516c40017e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:32.525204 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4525ac-2644-4168-8576-516c40017e14 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:32.525482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4525ac-2644-4168-8576-516c40017e14 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:32.525756 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4525ac-2644-4168-8576-516c40017e14 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:32.526433 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4525ac-2644-4168-8576-516c40017e14 None admin] RBAC: Adding query filter params (name=member) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:32.526735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4525ac-2644-4168-8576-516c40017e14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:32.528144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4525ac-2644-4168-8576-516c40017e14 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:32.538382 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 195/400] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:37:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:32.544430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f27e0bd-fe1a-474e-925b-d6b6f346246a None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:32.675744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f27e0bd-fe1a-474e-925b-d6b6f346246a None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:32.681975 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6a75a-b517-4bb4-a3da-9df6baa01b78 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:32.682576 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6a75a-b517-4bb4-a3da-9df6baa01b78 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:32.682576 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6a75a-b517-4bb4-a3da-9df6baa01b78 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:32.684231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 196/401] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:37:32.689428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b6161-bd9f-44f0-9ede-64d55ef2e023 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:32.689711 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b6161-bd9f-44f0-9ede-64d55ef2e023 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:32.690052 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b6161-bd9f-44f0-9ede-64d55ef2e023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:32.716699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-6e6b6161-bd9f-44f0-9ede-64d55ef2e023 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:32.718174 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6e6b6161-bd9f-44f0-9ede-64d55ef2e023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:32.722048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb7322-2c3d-4e95-9d1c-082b017ea41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:32.722710 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb7322-2c3d-4e95-9d1c-082b017ea41f None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:32.723013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb7322-2c3d-4e95-9d1c-082b017ea41f None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:32.723351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb7322-2c3d-4e95-9d1c-082b017ea41f None admin] PATH_INFO: `/v3/groups/8f8898e63cb545a5b96eb9c08232ba22` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:32.724927 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbb7322-2c3d-4e95-9d1c-082b017ea41f None admin] RBAC: Authorizing `identity:get_group(group_id=8f8898e63cb545a5b96eb9c08232ba22)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:32.726424 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbb7322-2c3d-4e95-9d1c-082b017ea41f None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:32.728171 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 206/402] 10.222.0.71 () {56 vars in 1180 bytes} [Fri Aug 26 08:37:32 2022] GET /identity/v3/groups/8f8898e63cb545a5b96eb9c08232ba22 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:32.734218 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb7322-2c3d-4e95-9d1c-082b017ea41f None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:32.764795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:32.807004 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 197/403] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 08:37:32.808414 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b6161-bd9f-44f0-9ede-64d55ef2e023 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:32.847603 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb7322-2c3d-4e95-9d1c-082b017ea41f None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:32.929188 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fc251-6275-4c1c-bb8f-bf88bbf32f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:32.929963 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fc251-6275-4c1c-bb8f-bf88bbf32f82 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:32.930248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fc251-6275-4c1c-bb8f-bf88bbf32f82 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:32.930524 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fc251-6275-4c1c-bb8f-bf88bbf32f82 None admin] PATH_INFO: `/v3/projects/214bc7d9ff5940989811937e08b70389` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:32.932137 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fc251-6275-4c1c-bb8f-bf88bbf32f82 None admin] RBAC: Authorizing `identity:get_project(project_id=214bc7d9ff5940989811937e08b70389)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:32.933811 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fc251-6275-4c1c-bb8f-bf88bbf32f82 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:32.940614 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 207/404] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:37:32 2022] GET /identity/v3/projects/214bc7d9ff5940989811937e08b70389 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:32.945945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fc251-6275-4c1c-bb8f-bf88bbf32f82 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:32.955526 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:33.022442 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b6161-bd9f-44f0-9ede-64d55ef2e023 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:33.063310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f7578-56e1-425f-9324-e5562b150e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:33.064130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f7578-56e1-425f-9324-e5562b150e19 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:33.064447 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f7578-56e1-425f-9324-e5562b150e19 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:33.064726 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f7578-56e1-425f-9324-e5562b150e19 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:33.078773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f7578-56e1-425f-9324-e5562b150e19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:33.085370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-821f7578-56e1-425f-9324-e5562b150e19 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:37:33.085370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:33.085370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:33.085370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:33.085370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:33.085370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:33.085370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:33.085370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:33.085370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:33.085370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:33.085370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:33.085370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:33.085370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:33.085370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:33.085370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:33.085370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:33.086310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:33.086310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:33.086310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:33.086310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:33.086310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:33.086310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:33.086310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:33.086310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:33.086310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:33.086310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:33.086310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:33.086310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:33.086310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:33.086310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:33.086310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:33.086310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:33.087123 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:33.087123 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:33.087123 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:33.087123 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:33.087123 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:33.087123 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:33.087123 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:33.087123 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:33.087123 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:33.087123 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:33.087123 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:37:33.087123 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:33.087123 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 198/405] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:37:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:33.094965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f7578-56e1-425f-9324-e5562b150e19 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:33.205271 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:33.222035 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fc251-6275-4c1c-bb8f-bf88bbf32f82 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:33.260593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f7578-56e1-425f-9324-e5562b150e19 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:33.285398 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3561c36-5d57-4884-b7a5-6d1ee095ed9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:33.286124 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3561c36-5d57-4884-b7a5-6d1ee095ed9a None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:33.286403 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3561c36-5d57-4884-b7a5-6d1ee095ed9a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:33.286676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3561c36-5d57-4884-b7a5-6d1ee095ed9a None admin] PATH_INFO: `/v3/projects/214bc7d9ff5940989811937e08b70389/groups/8f8898e63cb545a5b96eb9c08232ba22/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:33.288084 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:33.288939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3561c36-5d57-4884-b7a5-6d1ee095ed9a None admin] RBAC: Authorizing `identity:create_grant(project_id=214bc7d9ff5940989811937e08b70389, group_id=8f8898e63cb545a5b96eb9c08232ba22, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:33.291152 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3561c36-5d57-4884-b7a5-6d1ee095ed9a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:33.308767 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5243d6a3-fa68-4c86-a647-fb1cf0c0e1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:33.310144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243d6a3-fa68-4c86-a647-fb1cf0c0e1e5 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:33.310938 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243d6a3-fa68-4c86-a647-fb1cf0c0e1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:33.311548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243d6a3-fa68-4c86-a647-fb1cf0c0e1e5 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:33.311959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 208/406] 10.222.0.71 () {58 vars in 1442 bytes} [Fri Aug 26 08:37:32 2022] PUT /identity/v3/projects/214bc7d9ff5940989811937e08b70389/groups/8f8898e63cb545a5b96eb9c08232ba22/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:37:33.313127 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3561c36-5d57-4884-b7a5-6d1ee095ed9a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:33.318562 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5243d6a3-fa68-4c86-a647-fb1cf0c0e1e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:33.319630 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5243d6a3-fa68-4c86-a647-fb1cf0c0e1e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:33.321809 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5243d6a3-fa68-4c86-a647-fb1cf0c0e1e5 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:33.349004 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 199/407] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:37:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:33.355162 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5243d6a3-fa68-4c86-a647-fb1cf0c0e1e5 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:33.454170 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3561c36-5d57-4884-b7a5-6d1ee095ed9a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:33.501584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1cbfa-2587-4426-a108-f73814f472de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:33.502194 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1cbfa-2587-4426-a108-f73814f472de None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:33.502549 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1cbfa-2587-4426-a108-f73814f472de None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:33.502825 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1cbfa-2587-4426-a108-f73814f472de None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:33.530451 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1cbfa-2587-4426-a108-f73814f472de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:33.537762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-78c1cbfa-2587-4426-a108-f73814f472de None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 08:37:33.537762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:33.537762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:33.537762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:33.537762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:33.537762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:33.537762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:33.537762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:33.537762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:33.537762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:33.537762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:33.537762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:33.537762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:33.537762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:33.537762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:33.537762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:33.538725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:33.538725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:33.538725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:33.538725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:33.538725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:33.538725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:33.538725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:33.538725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:33.538725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:33.538725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:33.538725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:33.538725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:33.538725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:33.538725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:33.538725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:33.538725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:33.539696 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:33.539696 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:33.539696 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:33.539696 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:33.539696 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:33.539696 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:33.539696 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:33.539696 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:33.539696 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:33.539696 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:33.539696 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 08:37:33.539696 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:33.546466 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 209/408] 10.222.0.71 () {56 vars in 1114 bytes} [Fri Aug 26 08:37:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:33.559432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1cbfa-2587-4426-a108-f73814f472de None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:33.583035 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:33.602549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5243d6a3-fa68-4c86-a647-fb1cf0c0e1e5 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:33.655630 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f73d47-fb93-46c9-99b7-938cd1b975c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:33.656769 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f73d47-fb93-46c9-99b7-938cd1b975c1 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:33.658827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f73d47-fb93-46c9-99b7-938cd1b975c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:33.659516 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f73d47-fb93-46c9-99b7-938cd1b975c1 None admin] PATH_INFO: `/v3/groups/0380bf8970ea4d918c5197fb86dc6412` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:33.661077 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f73d47-fb93-46c9-99b7-938cd1b975c1 None admin] RBAC: Authorizing `identity:get_group(group_id=0380bf8970ea4d918c5197fb86dc6412)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:33.663034 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f73d47-fb93-46c9-99b7-938cd1b975c1 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:33.669849 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 200/409] 10.222.0.71 () {56 vars in 1180 bytes} [Fri Aug 26 08:37:33 2022] GET /identity/v3/groups/0380bf8970ea4d918c5197fb86dc6412 => generated 229 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:37:33.671608 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f73d47-fb93-46c9-99b7-938cd1b975c1 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:33.707350 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1cbfa-2587-4426-a108-f73814f472de None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:33.750078 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f029d4-2931-43d3-987a-1bf7fec02102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:33.750709 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f029d4-2931-43d3-987a-1bf7fec02102 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:33.751036 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f029d4-2931-43d3-987a-1bf7fec02102 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:33.751313 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f029d4-2931-43d3-987a-1bf7fec02102 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:33.752520 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f029d4-2931-43d3-987a-1bf7fec02102 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:33.752779 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f029d4-2931-43d3-987a-1bf7fec02102 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:33.754388 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f029d4-2931-43d3-987a-1bf7fec02102 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:33.764931 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 210/410] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:37:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:37:33.775643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f029d4-2931-43d3-987a-1bf7fec02102 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:33.807250 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:33.884557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f73d47-fb93-46c9-99b7-938cd1b975c1 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:33.916131 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:33.951607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf454d8-36f5-4785-9d49-38701b691bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:33.952427 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf454d8-36f5-4785-9d49-38701b691bee None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:33.952716 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf454d8-36f5-4785-9d49-38701b691bee None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:33.952988 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf454d8-36f5-4785-9d49-38701b691bee None admin] PATH_INFO: `/v3/projects/45132fb08c454ebaa1180c0b933c68b7` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:33.955562 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf454d8-36f5-4785-9d49-38701b691bee None admin] RBAC: Authorizing `identity:get_project(project_id=45132fb08c454ebaa1180c0b933c68b7)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:33.957066 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf454d8-36f5-4785-9d49-38701b691bee None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:33.965319 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 201/411] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:37:33 2022] GET /identity/v3/projects/45132fb08c454ebaa1180c0b933c68b7 => generated 344 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:37:33.967752 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f029d4-2931-43d3-987a-1bf7fec02102 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:33.970419 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf454d8-36f5-4785-9d49-38701b691bee None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:34.035226 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a40f9-e5d4-463c-b45b-62531679d9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:34.035796 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a40f9-e5d4-463c-b45b-62531679d9fc None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:34.036077 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a40f9-e5d4-463c-b45b-62531679d9fc None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:34.036351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a40f9-e5d4-463c-b45b-62531679d9fc None admin] PATH_INFO: `/v3/groups/8f8898e63cb545a5b96eb9c08232ba22` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:34.039304 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a40f9-e5d4-463c-b45b-62531679d9fc None admin] RBAC: Authorizing `identity:get_group(group_id=8f8898e63cb545a5b96eb9c08232ba22)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:34.040783 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a40f9-e5d4-463c-b45b-62531679d9fc None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:34.042573 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 211/412] 10.222.0.71 () {56 vars in 1180 bytes} [Fri Aug 26 08:37:33 2022] GET /identity/v3/groups/8f8898e63cb545a5b96eb9c08232ba22 => generated 226 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:37:34.049558 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a40f9-e5d4-463c-b45b-62531679d9fc None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:34.154233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf454d8-36f5-4785-9d49-38701b691bee None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:34.183319 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:34.200178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73e033-11ae-4274-b734-cf672709a11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:34.200897 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73e033-11ae-4274-b734-cf672709a11d None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:34.201172 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73e033-11ae-4274-b734-cf672709a11d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:34.201488 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73e033-11ae-4274-b734-cf672709a11d None admin] PATH_INFO: `/v3/projects/45132fb08c454ebaa1180c0b933c68b7/groups/0380bf8970ea4d918c5197fb86dc6412/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:34.204034 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc73e033-11ae-4274-b734-cf672709a11d None admin] RBAC: Authorizing `identity:create_grant(project_id=45132fb08c454ebaa1180c0b933c68b7, group_id=0380bf8970ea4d918c5197fb86dc6412, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:34.207634 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc73e033-11ae-4274-b734-cf672709a11d None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:34.230908 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 202/413] 10.222.0.71 () {58 vars in 1442 bytes} [Fri Aug 26 08:37:33 2022] PUT /identity/v3/projects/45132fb08c454ebaa1180c0b933c68b7/groups/0380bf8970ea4d918c5197fb86dc6412/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:37:34.246074 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a40f9-e5d4-463c-b45b-62531679d9fc None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:34.284998 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04205b33-89e0-45f6-8621-40ca2df76aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:34.285713 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04205b33-89e0-45f6-8621-40ca2df76aed None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:34.285933 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04205b33-89e0-45f6-8621-40ca2df76aed None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:34.286204 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04205b33-89e0-45f6-8621-40ca2df76aed None admin] PATH_INFO: `/v3/projects/214bc7d9ff5940989811937e08b70389` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:34.287445 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04205b33-89e0-45f6-8621-40ca2df76aed None admin] RBAC: Authorizing `identity:get_project(project_id=214bc7d9ff5940989811937e08b70389)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:34.288933 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04205b33-89e0-45f6-8621-40ca2df76aed None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:34.290583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 212/414] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:37:34 2022] GET /identity/v3/projects/214bc7d9ff5940989811937e08b70389 => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:37:34.296772 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73e033-11ae-4274-b734-cf672709a11d None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:34.403209 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8878719-6b60-4235-8262-20a71155799f None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:34.403209 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8878719-6b60-4235-8262-20a71155799f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:34.403209 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8878719-6b60-4235-8262-20a71155799f None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:34.403730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 213/415] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:34.410629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705abaf-d24a-4f75-a8cc-ef61b75cff06 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:34.412426 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705abaf-d24a-4f75-a8cc-ef61b75cff06 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:34.412426 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705abaf-d24a-4f75-a8cc-ef61b75cff06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:34.434593 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-0705abaf-d24a-4f75-a8cc-ef61b75cff06 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:34.440384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-0705abaf-d24a-4f75-a8cc-ef61b75cff06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:34.459089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:34.462348 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73e033-11ae-4274-b734-cf672709a11d None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:34.547731 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c043b7-9071-4d55-99a1-f35e618e83dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:34.549904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c043b7-9071-4d55-99a1-f35e618e83dd None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:34.550374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c043b7-9071-4d55-99a1-f35e618e83dd None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:34.551062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c043b7-9071-4d55-99a1-f35e618e83dd None admin] PATH_INFO: `/v3/projects/214bc7d9ff5940989811937e08b70389/groups/8f8898e63cb545a5b96eb9c08232ba22/roles/6195dc21ae4749daa31797d2c42e050e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:34.556122 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c043b7-9071-4d55-99a1-f35e618e83dd None admin] RBAC: Authorizing `identity:create_grant(project_id=214bc7d9ff5940989811937e08b70389, group_id=8f8898e63cb545a5b96eb9c08232ba22, role_id=6195dc21ae4749daa31797d2c42e050e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:34.558784 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 214/416] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 08:37:34.568068 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c043b7-9071-4d55-99a1-f35e618e83dd None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:34.592185 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 203/417] 10.222.0.71 () {58 vars in 1442 bytes} [Fri Aug 26 08:37:34 2022] PUT /identity/v3/projects/214bc7d9ff5940989811937e08b70389/groups/8f8898e63cb545a5b96eb9c08232ba22/roles/6195dc21ae4749daa31797d2c42e050e => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:37:34.650317 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91edd170-7f68-408e-8965-ef8471ef83f2 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:34.650675 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91edd170-7f68-408e-8965-ef8471ef83f2 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:34.650869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91edd170-7f68-408e-8965-ef8471ef83f2 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:34.652609 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 215/418] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:34.659763 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c44cf-d55e-4cb8-a372-70f47a765266 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:34.660256 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c44cf-d55e-4cb8-a372-70f47a765266 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:34.660740 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c44cf-d55e-4cb8-a372-70f47a765266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:34.692382 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-bf8c44cf-d55e-4cb8-a372-70f47a765266 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:34.693692 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-bf8c44cf-d55e-4cb8-a372-70f47a765266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:34.732115 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:34.781049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 204/419] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:34.904664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91edd170-7f68-408e-8965-ef8471ef83f2 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:35.043406 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91edd170-7f68-408e-8965-ef8471ef83f2 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:35.100008 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731590ff-c77d-4f22-8706-858c1e5ec854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:35.100851 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731590ff-c77d-4f22-8706-858c1e5ec854 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:35.101155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731590ff-c77d-4f22-8706-858c1e5ec854 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:35.101437 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731590ff-c77d-4f22-8706-858c1e5ec854 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:35.117808 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731590ff-c77d-4f22-8706-858c1e5ec854 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:35.123329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-731590ff-c77d-4f22-8706-858c1e5ec854 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:37:35.123329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:35.123329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:35.123329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:35.123329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:35.123329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:35.123329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:35.123329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:35.123329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:35.123329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:35.123329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:35.123329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:35.123329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:35.123329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:35.123329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:35.123329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:35.124333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:35.124333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:35.124333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:35.124333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:35.124333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:35.124333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:35.124333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:35.124333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:35.124333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:35.124333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:35.124333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:35.124333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:35.124333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:35.124333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:35.124333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:35.124333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:35.125144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:35.125144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:35.125144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:35.125144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:35.125144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:35.125144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:35.125144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:35.125144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:35.125144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:35.125144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:35.125144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:37:35.125144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:35.125144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 216/420] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:37:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:35.136113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c44cf-d55e-4cb8-a372-70f47a765266 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:35.177465 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731590ff-c77d-4f22-8706-858c1e5ec854 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:35.192086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:35.341303 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c44cf-d55e-4cb8-a372-70f47a765266 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:35.365115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731590ff-c77d-4f22-8706-858c1e5ec854 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:35.407706 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa4833-2f86-474c-a89c-6ec1422927aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:35.408620 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa4833-2f86-474c-a89c-6ec1422927aa None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:35.409084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa4833-2f86-474c-a89c-6ec1422927aa None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:35.409659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa4833-2f86-474c-a89c-6ec1422927aa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:35.423316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa4833-2f86-474c-a89c-6ec1422927aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:35.430055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-35fa4833-2f86-474c-a89c-6ec1422927aa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 08:37:35.430055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:35.430055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:35.430055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:35.430055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:35.430055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:35.430055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:35.430055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:35.430055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:35.430055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:35.430055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:35.430055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:35.430055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:35.430055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:35.430055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:35.430055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:35.431237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:35.431237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:35.431237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:35.431237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:35.431237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:35.431237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:35.431237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:35.431237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:35.431237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:35.431237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:35.431237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:35.431237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:35.431237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:35.431237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:35.431237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:35.431237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:35.432238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:35.432238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:35.432238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:35.432238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:35.432238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:35.432238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:35.432238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:35.432238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:35.432238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:35.432238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:35.432238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 08:37:35.432238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:35.434598 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 217/421] 10.222.0.71 () {56 vars in 1114 bytes} [Fri Aug 26 08:37:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:37:35.443999 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa4833-2f86-474c-a89c-6ec1422927aa None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:35.451710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06d0e4-36f2-470e-9f98-1c1f91fc1430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:35.452354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06d0e4-36f2-470e-9f98-1c1f91fc1430 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:35.452638 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06d0e4-36f2-470e-9f98-1c1f91fc1430 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:35.452914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06d0e4-36f2-470e-9f98-1c1f91fc1430 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:35.453620 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f06d0e4-36f2-470e-9f98-1c1f91fc1430 None admin] RBAC: Adding query filter params (name=member) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:35.453928 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f06d0e4-36f2-470e-9f98-1c1f91fc1430 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:35.455906 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f06d0e4-36f2-470e-9f98-1c1f91fc1430 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:35.484089 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 205/422] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:37:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:35.490819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06d0e4-36f2-470e-9f98-1c1f91fc1430 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:35.534767 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:35.569853 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:35.586622 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa4833-2f86-474c-a89c-6ec1422927aa None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:35.639988 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b48286-1173-4289-b903-7e65d6bde8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:35.642942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b48286-1173-4289-b903-7e65d6bde8f6 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:35.643650 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b48286-1173-4289-b903-7e65d6bde8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:35.644603 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b48286-1173-4289-b903-7e65d6bde8f6 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:35.647148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b48286-1173-4289-b903-7e65d6bde8f6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:35.648210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b48286-1173-4289-b903-7e65d6bde8f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:35.652190 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b48286-1173-4289-b903-7e65d6bde8f6 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:35.672236 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 218/423] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:37:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:35.680281 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b48286-1173-4289-b903-7e65d6bde8f6 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:35.736370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06d0e4-36f2-470e-9f98-1c1f91fc1430 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:35.815999 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b48286-1173-4289-b903-7e65d6bde8f6 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:35.823830 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952f469-81d6-4c0a-962c-3816b23406b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:35.824574 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952f469-81d6-4c0a-962c-3816b23406b2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:35.824855 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952f469-81d6-4c0a-962c-3816b23406b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:35.825130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952f469-81d6-4c0a-962c-3816b23406b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:35.851326 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:35.870467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21515f36-f7e1-4794-b59a-8ca3c7e6a8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:35.872119 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21515f36-f7e1-4794-b59a-8ca3c7e6a8d0 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:35.872872 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21515f36-f7e1-4794-b59a-8ca3c7e6a8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:35.873610 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21515f36-f7e1-4794-b59a-8ca3c7e6a8d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:35.949931 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952f469-81d6-4c0a-962c-3816b23406b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:35.952782 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952f469-81d6-4c0a-962c-3816b23406b2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:35.990821 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21515f36-f7e1-4794-b59a-8ca3c7e6a8d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:35.993371 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21515f36-f7e1-4794-b59a-8ca3c7e6a8d0 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:35.997697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:36.103164 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 206/424] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:37:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:37:36.104842 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef757df-f94c-4217-89df-36674863a66f None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:36.105342 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef757df-f94c-4217-89df-36674863a66f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:36.105808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef757df-f94c-4217-89df-36674863a66f None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:36.107862 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 207/425] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:36.108901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef757df-f94c-4217-89df-36674863a66f None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:36.147384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 219/426] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:37:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:37:36.149431 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ea2f3-2c2f-439a-822c-f9fef56ba64f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:36.150014 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ea2f3-2c2f-439a-822c-f9fef56ba64f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:36.150558 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ea2f3-2c2f-439a-822c-f9fef56ba64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:36.176302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-795ea2f3-2c2f-439a-822c-f9fef56ba64f None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:36.177769 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-795ea2f3-2c2f-439a-822c-f9fef56ba64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:36.205705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:36.255853 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef757df-f94c-4217-89df-36674863a66f None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:36.284867 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 220/427] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:36.286094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ea2f3-2c2f-439a-822c-f9fef56ba64f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:36.315490 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:36.335699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4de67-d81c-4dd7-ab97-6231696210d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:36.336388 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4de67-d81c-4dd7-ab97-6231696210d8 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:36.336657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4de67-d81c-4dd7-ab97-6231696210d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:36.336939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4de67-d81c-4dd7-ab97-6231696210d8 None admin] PATH_INFO: `/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:36.338212 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc4de67-d81c-4dd7-ab97-6231696210d8 None admin] RBAC: Authorizing `identity:get_project(project_id=e55f1cc9f9c9481698d6962c7af7fbfe)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:36.339706 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc4de67-d81c-4dd7-ab97-6231696210d8 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:36.341826 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 208/428] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:37:36 2022] GET /identity/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:36.350762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4de67-d81c-4dd7-ab97-6231696210d8 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:36.490199 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ea2f3-2c2f-439a-822c-f9fef56ba64f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:36.563867 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5fb3c0-964d-4792-9cff-98ef57f13ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:36.564543 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fb3c0-964d-4792-9cff-98ef57f13ef3 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:36.564817 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fb3c0-964d-4792-9cff-98ef57f13ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:36.565173 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5fb3c0-964d-4792-9cff-98ef57f13ef3 None admin] PATH_INFO: `/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:36.567968 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5fb3c0-964d-4792-9cff-98ef57f13ef3 None admin] RBAC: Authorizing `identity:get_project(project_id=e55f1cc9f9c9481698d6962c7af7fbfe)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:36.571435 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5fb3c0-964d-4792-9cff-98ef57f13ef3 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:36.573282 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 221/429] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:37:36 2022] GET /identity/v3/projects/e55f1cc9f9c9481698d6962c7af7fbfe => generated 298 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:36.577434 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4de67-d81c-4dd7-ab97-6231696210d8 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:36.580955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5fb3c0-964d-4792-9cff-98ef57f13ef3 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:36.632709 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:36.655051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af3891-4c4d-4b4f-ad5a-ba6ce470aa5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:36.661197 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af3891-4c4d-4b4f-ad5a-ba6ce470aa5d None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:36.661700 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af3891-4c4d-4b4f-ad5a-ba6ce470aa5d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:36.662112 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af3891-4c4d-4b4f-ad5a-ba6ce470aa5d None admin] PATH_INFO: `/v3/groups/8f8898e63cb545a5b96eb9c08232ba22` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:36.665718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af3891-4c4d-4b4f-ad5a-ba6ce470aa5d None admin] RBAC: Authorizing `identity:get_group(group_id=8f8898e63cb545a5b96eb9c08232ba22)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:36.667675 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af3891-4c4d-4b4f-ad5a-ba6ce470aa5d None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:36.669786 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 209/430] 10.222.0.71 () {56 vars in 1180 bytes} [Fri Aug 26 08:37:36 2022] GET /identity/v3/groups/8f8898e63cb545a5b96eb9c08232ba22 => generated 226 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:36.670989 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af3891-4c4d-4b4f-ad5a-ba6ce470aa5d None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:36.752329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5fb3c0-964d-4792-9cff-98ef57f13ef3 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:36.764341 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:36.808697 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8337fd2-1701-4622-8b13-8572ab804461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:36.809636 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8337fd2-1701-4622-8b13-8572ab804461 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:36.809978 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8337fd2-1701-4622-8b13-8572ab804461 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:36.810275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8337fd2-1701-4622-8b13-8572ab804461 None admin] PATH_INFO: `/v3/groups/8f8898e63cb545a5b96eb9c08232ba22` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:36.812468 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8337fd2-1701-4622-8b13-8572ab804461 None admin] RBAC: Authorizing `identity:get_group(group_id=8f8898e63cb545a5b96eb9c08232ba22)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:36.815906 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8337fd2-1701-4622-8b13-8572ab804461 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:36.819028 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af3891-4c4d-4b4f-ad5a-ba6ce470aa5d None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:36.819324 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 222/431] 10.222.0.71 () {56 vars in 1180 bytes} [Fri Aug 26 08:37:36 2022] GET /identity/v3/groups/8f8898e63cb545a5b96eb9c08232ba22 => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:36.820565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8337fd2-1701-4622-8b13-8572ab804461 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:36.868097 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49413bf-48dd-4300-adae-c4fae47ef292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:36.869049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49413bf-48dd-4300-adae-c4fae47ef292 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:36.869571 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49413bf-48dd-4300-adae-c4fae47ef292 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:36.870011 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49413bf-48dd-4300-adae-c4fae47ef292 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:36.886685 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49413bf-48dd-4300-adae-c4fae47ef292 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:36.896454 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-e49413bf-48dd-4300-adae-c4fae47ef292 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:37:36.896454 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:36.896454 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:36.896454 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:36.896454 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:36.896454 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:36.896454 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:36.896454 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:36.896454 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:36.896454 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:36.896454 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:36.896454 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:36.896454 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:36.896454 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:36.896454 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:36.896454 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:36.897874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:36.897874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:36.897874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:36.897874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:36.897874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:36.897874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:36.897874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:36.897874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:36.897874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:36.897874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:36.897874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:36.897874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:36.897874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:36.897874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:36.897874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:36.897874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:36.898796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:36.898796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:36.898796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:36.898796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:36.898796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:36.898796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:36.898796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:36.898796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:36.898796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:36.898796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:36.898796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 08:37:36.898796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:36.901350 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 210/432] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:37:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:36.908504 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49413bf-48dd-4300-adae-c4fae47ef292 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:36.952181 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:36.968560 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8337fd2-1701-4622-8b13-8572ab804461 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:37.039483 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f5bd1-42ec-4ad5-9c7c-f1a892397504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:37.040528 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49413bf-48dd-4300-adae-c4fae47ef292 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:37.042378 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f5bd1-42ec-4ad5-9c7c-f1a892397504 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:37.043171 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f5bd1-42ec-4ad5-9c7c-f1a892397504 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:37.043899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f5bd1-42ec-4ad5-9c7c-f1a892397504 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:37.045654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f5bd1-42ec-4ad5-9c7c-f1a892397504 None admin] RBAC: Adding query filter params (group.id=8f8898e63cb545a5b96eb9c08232ba22, role.id=b7a48a25b0cd479d85fe5395d64c458e, scope.project.id=e55f1cc9f9c9481698d6962c7af7fbfe) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:37.046960 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f5bd1-42ec-4ad5-9c7c-f1a892397504 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:37.051584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f5bd1-42ec-4ad5-9c7c-f1a892397504 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:37.065369 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 223/433] 10.222.0.71 () {56 vars in 1376 bytes} [Fri Aug 26 08:37:36 2022] GET /identity/v3/role_assignments?group.id=8f8898e63cb545a5b96eb9c08232ba22&scope.project.id=e55f1cc9f9c9481698d6962c7af7fbfe&role.id=b7a48a25b0cd479d85fe5395d64c458e => generated 611 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:37.067687 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f5bd1-42ec-4ad5-9c7c-f1a892397504 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:37.071979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:37.082442 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e729a83-7ef1-481c-80a4-735e922f9402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:37.083136 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e729a83-7ef1-481c-80a4-735e922f9402 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:37.083410 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e729a83-7ef1-481c-80a4-735e922f9402 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:37.083690 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e729a83-7ef1-481c-80a4-735e922f9402 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:37.084636 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e729a83-7ef1-481c-80a4-735e922f9402 None admin] RBAC: Adding query filter params (group.id=8f8898e63cb545a5b96eb9c08232ba22, role.id=6195dc21ae4749daa31797d2c42e050e, scope.project.id=e55f1cc9f9c9481698d6962c7af7fbfe) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:37.084937 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e729a83-7ef1-481c-80a4-735e922f9402 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:37.086697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e729a83-7ef1-481c-80a4-735e922f9402 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:37.095341 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 211/434] 10.222.0.71 () {56 vars in 1376 bytes} [Fri Aug 26 08:37:36 2022] GET /identity/v3/role_assignments?group.id=8f8898e63cb545a5b96eb9c08232ba22&scope.project.id=e55f1cc9f9c9481698d6962c7af7fbfe&role.id=6195dc21ae4749daa31797d2c42e050e => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:37.120299 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf7aa8-5d3b-4048-8777-bb60edb46917 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:37.120848 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf7aa8-5d3b-4048-8777-bb60edb46917 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:37.121325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf7aa8-5d3b-4048-8777-bb60edb46917 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:37.122991 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 212/435] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:37.129840 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25708e5-be44-4a7b-ab52-d116a0296db7 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:37.130301 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25708e5-be44-4a7b-ab52-d116a0296db7 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:37.130673 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25708e5-be44-4a7b-ab52-d116a0296db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:37.154191 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-d25708e5-be44-4a7b-ab52-d116a0296db7 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:37.161277 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d25708e5-be44-4a7b-ab52-d116a0296db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:37.283911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f5bd1-42ec-4ad5-9c7c-f1a892397504 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:37.287711 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 213/436] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 08:37:37.371023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1926e5-ba6f-454d-b6b4-02847c335da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:37.371632 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1926e5-ba6f-454d-b6b4-02847c335da0 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:37.371914 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1926e5-ba6f-454d-b6b4-02847c335da0 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:37.372182 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1926e5-ba6f-454d-b6b4-02847c335da0 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:37.373222 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1926e5-ba6f-454d-b6b4-02847c335da0 None admin] RBAC: Adding query filter params (name=member) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:37.373222 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1926e5-ba6f-454d-b6b4-02847c335da0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:37.374606 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1926e5-ba6f-454d-b6b4-02847c335da0 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:37.386013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 224/437] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:37:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:37.392229 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25708e5-be44-4a7b-ab52-d116a0296db7 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:37.535051 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244fbfb-9400-49bd-bfb2-7d4d7bb6a924 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:37.535751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244fbfb-9400-49bd-bfb2-7d4d7bb6a924 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:37.536471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244fbfb-9400-49bd-bfb2-7d4d7bb6a924 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:37.539088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 225/438] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:37.547699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180fc3e-cbba-455c-9aad-68ce9d584235 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:37.548363 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180fc3e-cbba-455c-9aad-68ce9d584235 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:37.548655 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180fc3e-cbba-455c-9aad-68ce9d584235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:37.572375 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-1180fc3e-cbba-455c-9aad-68ce9d584235 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:37.573693 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1180fc3e-cbba-455c-9aad-68ce9d584235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:37.597334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:37.600058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25708e5-be44-4a7b-ab52-d116a0296db7 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:37.639410 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:37.670397 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a41d8e-284f-40a9-80d0-b76256739c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:37.671082 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a41d8e-284f-40a9-80d0-b76256739c14 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:37.671380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a41d8e-284f-40a9-80d0-b76256739c14 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:37.671646 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a41d8e-284f-40a9-80d0-b76256739c14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:37.728791 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 226/439] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 08:37:37.730578 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180fc3e-cbba-455c-9aad-68ce9d584235 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:37.775862 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a41d8e-284f-40a9-80d0-b76256739c14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:37.778652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a41d8e-284f-40a9-80d0-b76256739c14 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:37.947944 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180fc3e-cbba-455c-9aad-68ce9d584235 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:37.973439 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 214/440] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:37:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:37:37.982020 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a41d8e-284f-40a9-80d0-b76256739c14 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:38.017751 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:38.028318 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6095d-646e-4a48-a8f5-a144130cd900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:38.029000 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6095d-646e-4a48-a8f5-a144130cd900 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:38.029291 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6095d-646e-4a48-a8f5-a144130cd900 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:38.029573 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6095d-646e-4a48-a8f5-a144130cd900 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:38.053434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d6095d-646e-4a48-a8f5-a144130cd900 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:38.062302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-05d6095d-646e-4a48-a8f5-a144130cd900 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:37:38.062302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:38.062302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:38.062302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:38.062302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:38.062302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:38.062302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:38.062302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:38.062302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:38.062302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:38.062302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:38.062302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:38.062302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:38.062302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:38.062302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:38.062302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:38.064584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:38.064584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:38.064584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:38.064584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:38.064584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:38.064584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:38.064584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:38.064584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:38.064584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:38.064584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:38.064584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:38.064584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:38.064584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:38.064584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:38.064584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:38.064584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:38.066652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:38.066652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:38.066652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:38.066652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:38.066652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:38.066652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:38.066652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:38.066652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:38.066652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:38.066652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:38.066652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:37:38.066652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:38.066652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 227/441] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:37:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:38.066652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6095d-646e-4a48-a8f5-a144130cd900 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:38.083339 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:38.190233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a41d8e-284f-40a9-80d0-b76256739c14 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:38.232232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6095d-646e-4a48-a8f5-a144130cd900 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:38.249684 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e92a9f-8d96-4866-928d-a01a6652bfbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:38.250317 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e92a9f-8d96-4866-928d-a01a6652bfbc None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:38.250600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e92a9f-8d96-4866-928d-a01a6652bfbc None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:38.250873 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e92a9f-8d96-4866-928d-a01a6652bfbc None admin] PATH_INFO: `/v3/projects/214bc7d9ff5940989811937e08b70389` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:38.252911 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e92a9f-8d96-4866-928d-a01a6652bfbc None admin] RBAC: Authorizing `identity:get_project(project_id=214bc7d9ff5940989811937e08b70389)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:38.255563 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e92a9f-8d96-4866-928d-a01a6652bfbc None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:38.255563 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 215/442] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:37:37 2022] GET /identity/v3/projects/214bc7d9ff5940989811937e08b70389 => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:38.255563 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e92a9f-8d96-4866-928d-a01a6652bfbc None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:38.286305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854e215-59d6-4298-9107-0e5d18d50190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:38.287065 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854e215-59d6-4298-9107-0e5d18d50190 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:38.287338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854e215-59d6-4298-9107-0e5d18d50190 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:38.287615 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854e215-59d6-4298-9107-0e5d18d50190 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:38.305817 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854e215-59d6-4298-9107-0e5d18d50190 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:38.310815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-e854e215-59d6-4298-9107-0e5d18d50190 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 08:37:38.310815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:38.310815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:38.310815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:38.310815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:38.310815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:38.310815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:38.310815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:38.310815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:38.310815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:38.310815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:38.310815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:38.310815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:38.310815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:38.310815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:38.310815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:38.313942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:38.313942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:38.313942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:38.313942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:38.313942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:38.313942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:38.313942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:38.313942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:38.313942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:38.313942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:38.313942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:38.313942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:38.313942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:38.313942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:38.313942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:38.313942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:38.314911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:38.314911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:38.314911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:38.314911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:38.314911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:38.314911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:38.314911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:38.314911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:38.314911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:38.314911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:38.314911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 08:37:38.314911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:38.314911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 228/443] 10.222.0.71 () {56 vars in 1114 bytes} [Fri Aug 26 08:37:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:38.314911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854e215-59d6-4298-9107-0e5d18d50190 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:38.402937 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:38.405602 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:38.418146 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e92a9f-8d96-4866-928d-a01a6652bfbc None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:38.450225 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854e215-59d6-4298-9107-0e5d18d50190 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:38.472653 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeca7a5-9f9d-4752-bcf7-08509845fa52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:38.473451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeca7a5-9f9d-4752-bcf7-08509845fa52 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:38.473913 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeca7a5-9f9d-4752-bcf7-08509845fa52 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:38.474288 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeca7a5-9f9d-4752-bcf7-08509845fa52 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:38.475176 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeca7a5-9f9d-4752-bcf7-08509845fa52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:38.475625 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeca7a5-9f9d-4752-bcf7-08509845fa52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:38.477397 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeca7a5-9f9d-4752-bcf7-08509845fa52 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:38.489154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 216/444] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:37:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:38.489790 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeca7a5-9f9d-4752-bcf7-08509845fa52 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:38.514783 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816d64d-b01f-430b-b15e-4cd42e23df12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:38.515972 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816d64d-b01f-430b-b15e-4cd42e23df12 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:38.516417 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816d64d-b01f-430b-b15e-4cd42e23df12 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:38.516902 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816d64d-b01f-430b-b15e-4cd42e23df12 None admin] PATH_INFO: `/v3/groups/8f8898e63cb545a5b96eb9c08232ba22` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:38.519497 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816d64d-b01f-430b-b15e-4cd42e23df12 None admin] RBAC: Authorizing `identity:get_group(group_id=8f8898e63cb545a5b96eb9c08232ba22)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:38.522422 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816d64d-b01f-430b-b15e-4cd42e23df12 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:38.530362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 229/445] 10.222.0.71 () {56 vars in 1180 bytes} [Fri Aug 26 08:37:38 2022] GET /identity/v3/groups/8f8898e63cb545a5b96eb9c08232ba22 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:38.530362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816d64d-b01f-430b-b15e-4cd42e23df12 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:38.613099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeca7a5-9f9d-4752-bcf7-08509845fa52 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:38.653690 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816d64d-b01f-430b-b15e-4cd42e23df12 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:38.661238 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ce212-171c-491f-8107-bc3bd70b54f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:38.662833 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ce212-171c-491f-8107-bc3bd70b54f5 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:38.662833 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ce212-171c-491f-8107-bc3bd70b54f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:38.662833 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ce212-171c-491f-8107-bc3bd70b54f5 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:38.663438 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ce212-171c-491f-8107-bc3bd70b54f5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:38.663773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ce212-171c-491f-8107-bc3bd70b54f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:38.665227 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ce212-171c-491f-8107-bc3bd70b54f5 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:38.674457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 217/446] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:37:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:38.675380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ce212-171c-491f-8107-bc3bd70b54f5 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:38.698238 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:38.703860 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f7988-4891-4f11-9a6f-65438e4478c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:38.704500 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f7988-4891-4f11-9a6f-65438e4478c1 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:38.704770 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f7988-4891-4f11-9a6f-65438e4478c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:38.705048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f7988-4891-4f11-9a6f-65438e4478c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:38.708842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:38.800061 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4f7988-4891-4f11-9a6f-65438e4478c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:38.801751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4f7988-4891-4f11-9a6f-65438e4478c1 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:38.817293 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ce212-171c-491f-8107-bc3bd70b54f5 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:38.863170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced3530-26c6-4512-af0a-59216e43c1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:38.863995 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced3530-26c6-4512-af0a-59216e43c1ca None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:38.864074 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced3530-26c6-4512-af0a-59216e43c1ca None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:38.864331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced3530-26c6-4512-af0a-59216e43c1ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:38.865193 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced3530-26c6-4512-af0a-59216e43c1ca None admin] RBAC: Adding query filter params (group.id=8f8898e63cb545a5b96eb9c08232ba22, role.id=b7a48a25b0cd479d85fe5395d64c458e, scope.project.id=214bc7d9ff5940989811937e08b70389) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:38.865382 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced3530-26c6-4512-af0a-59216e43c1ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:38.867196 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced3530-26c6-4512-af0a-59216e43c1ca None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:38.873887 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 218/447] 10.222.0.71 () {56 vars in 1376 bytes} [Fri Aug 26 08:37:38 2022] GET /identity/v3/role_assignments?group.id=8f8898e63cb545a5b96eb9c08232ba22&scope.project.id=214bc7d9ff5940989811937e08b70389&role.id=b7a48a25b0cd479d85fe5395d64c458e => generated 611 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:38.874740 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced3530-26c6-4512-af0a-59216e43c1ca None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:38.960514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 230/448] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:37:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:37:38.961952 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f7988-4891-4f11-9a6f-65438e4478c1 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:39.016524 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:39.028942 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:39.059803 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced3530-26c6-4512-af0a-59216e43c1ca None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:39.108872 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a790c-ddfe-4845-a663-a5763c74f519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:39.108872 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a790c-ddfe-4845-a663-a5763c74f519 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:39.109259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a790c-ddfe-4845-a663-a5763c74f519 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:39.109403 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a790c-ddfe-4845-a663-a5763c74f519 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:39.114541 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f7988-4891-4f11-9a6f-65438e4478c1 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:39.171154 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a24d6c-071b-4b08-bc42-f597139403d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:39.171848 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a24d6c-071b-4b08-bc42-f597139403d6 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:39.172136 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a24d6c-071b-4b08-bc42-f597139403d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:39.172407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a24d6c-071b-4b08-bc42-f597139403d6 None admin] PATH_INFO: `/v3/projects/45132fb08c454ebaa1180c0b933c68b7` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:39.174456 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a24d6c-071b-4b08-bc42-f597139403d6 None admin] RBAC: Authorizing `identity:get_project(project_id=45132fb08c454ebaa1180c0b933c68b7)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:39.176360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a24d6c-071b-4b08-bc42-f597139403d6 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:39.177997 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 231/449] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:37:38 2022] GET /identity/v3/projects/45132fb08c454ebaa1180c0b933c68b7 => generated 344 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:39.187204 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a24d6c-071b-4b08-bc42-f597139403d6 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:39.221813 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a790c-ddfe-4845-a663-a5763c74f519 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:39.223463 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a790c-ddfe-4845-a663-a5763c74f519 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:39.309640 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a24d6c-071b-4b08-bc42-f597139403d6 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:39.311280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:39.321729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:39.345981 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9431881-6e83-471f-b51e-be59fb0e3f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:39.351474 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 219/450] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:37:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:37:39.351712 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9431881-6e83-471f-b51e-be59fb0e3f2f None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:39.351712 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9431881-6e83-471f-b51e-be59fb0e3f2f None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:39.351712 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9431881-6e83-471f-b51e-be59fb0e3f2f None admin] PATH_INFO: `/v3/groups/0380bf8970ea4d918c5197fb86dc6412` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:39.351712 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9431881-6e83-471f-b51e-be59fb0e3f2f None admin] RBAC: Authorizing `identity:get_group(group_id=0380bf8970ea4d918c5197fb86dc6412)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:39.351712 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9431881-6e83-471f-b51e-be59fb0e3f2f None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:39.351712 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 232/451] 10.222.0.71 () {56 vars in 1180 bytes} [Fri Aug 26 08:37:39 2022] GET /identity/v3/groups/0380bf8970ea4d918c5197fb86dc6412 => generated 229 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:37:39.352539 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a790c-ddfe-4845-a663-a5763c74f519 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:39.357561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9431881-6e83-471f-b51e-be59fb0e3f2f None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:39.460777 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a790c-ddfe-4845-a663-a5763c74f519 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:39.470398 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9431881-6e83-471f-b51e-be59fb0e3f2f None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:39.497226 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256547df-a388-477c-a405-03b613bd98e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:39.497864 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256547df-a388-477c-a405-03b613bd98e7 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:39.498137 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256547df-a388-477c-a405-03b613bd98e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:39.498396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256547df-a388-477c-a405-03b613bd98e7 None admin] PATH_INFO: `/v3/projects/214bc7d9ff5940989811937e08b70389` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:39.500129 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256547df-a388-477c-a405-03b613bd98e7 None admin] RBAC: Authorizing `identity:get_project(project_id=214bc7d9ff5940989811937e08b70389)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:39.501989 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256547df-a388-477c-a405-03b613bd98e7 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:39.503943 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 220/452] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:37:39 2022] GET /identity/v3/projects/214bc7d9ff5940989811937e08b70389 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:39.507624 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955229fa-fae9-4713-83b6-356ad419382e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:39.508253 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955229fa-fae9-4713-83b6-356ad419382e None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:39.508524 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955229fa-fae9-4713-83b6-356ad419382e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:39.508788 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955229fa-fae9-4713-83b6-356ad419382e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:39.509537 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955229fa-fae9-4713-83b6-356ad419382e None admin] RBAC: Adding query filter params (group.id=0380bf8970ea4d918c5197fb86dc6412, role.id=18c3c8a821b647f7982a955880bd0a9a, scope.project.id=45132fb08c454ebaa1180c0b933c68b7) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:39.509836 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955229fa-fae9-4713-83b6-356ad419382e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:39.510444 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256547df-a388-477c-a405-03b613bd98e7 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:39.511314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955229fa-fae9-4713-83b6-356ad419382e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:39.518067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 233/453] 10.222.0.71 () {56 vars in 1376 bytes} [Fri Aug 26 08:37:39 2022] GET /identity/v3/role_assignments?group.id=0380bf8970ea4d918c5197fb86dc6412&scope.project.id=45132fb08c454ebaa1180c0b933c68b7&role.id=18c3c8a821b647f7982a955880bd0a9a => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:39.569354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:39.645081 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256547df-a388-477c-a405-03b613bd98e7 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:39.682908 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e3f00-521a-4ea3-a472-675a403420aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:39.683546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e3f00-521a-4ea3-a472-675a403420aa None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:39.683892 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e3f00-521a-4ea3-a472-675a403420aa None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:39.684092 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e3f00-521a-4ea3-a472-675a403420aa None admin] PATH_INFO: `/v3/groups/8f8898e63cb545a5b96eb9c08232ba22` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:39.685265 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e3f00-521a-4ea3-a472-675a403420aa None admin] RBAC: Authorizing `identity:get_group(group_id=8f8898e63cb545a5b96eb9c08232ba22)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:39.686716 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e3f00-521a-4ea3-a472-675a403420aa None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:39.688361 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 221/454] 10.222.0.71 () {56 vars in 1180 bytes} [Fri Aug 26 08:37:39 2022] GET /identity/v3/groups/8f8898e63cb545a5b96eb9c08232ba22 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:37:39.694607 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955229fa-fae9-4713-83b6-356ad419382e None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:39.731031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:39.801329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955229fa-fae9-4713-83b6-356ad419382e None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:39.842397 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff6eb6-fcff-4e33-aa95-b22bf64ca6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:39.843251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff6eb6-fcff-4e33-aa95-b22bf64ca6c1 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:39.843703 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff6eb6-fcff-4e33-aa95-b22bf64ca6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:39.844131 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff6eb6-fcff-4e33-aa95-b22bf64ca6c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:39.845236 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daff6eb6-fcff-4e33-aa95-b22bf64ca6c1 None admin] RBAC: Adding query filter params (group.id=8f8898e63cb545a5b96eb9c08232ba22, role.id=6195dc21ae4749daa31797d2c42e050e, scope.project.id=214bc7d9ff5940989811937e08b70389) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:39.845731 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daff6eb6-fcff-4e33-aa95-b22bf64ca6c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:39.847992 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daff6eb6-fcff-4e33-aa95-b22bf64ca6c1 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:39.858235 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 234/455] 10.222.0.71 () {56 vars in 1376 bytes} [Fri Aug 26 08:37:39 2022] GET /identity/v3/role_assignments?group.id=8f8898e63cb545a5b96eb9c08232ba22&scope.project.id=214bc7d9ff5940989811937e08b70389&role.id=6195dc21ae4749daa31797d2c42e050e => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:42.450668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54fcaaa-b166-4a2f-81b2-1814bf4d8c94 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:42.451207 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54fcaaa-b166-4a2f-81b2-1814bf4d8c94 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:42.451281 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54fcaaa-b166-4a2f-81b2-1814bf4d8c94 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:42.452751 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 222/456] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:42.462908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263816b-8ace-411d-b407-372915f0e09e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:42.462908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263816b-8ace-411d-b407-372915f0e09e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:42.462908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6263816b-8ace-411d-b407-372915f0e09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:42.488183 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-6263816b-8ace-411d-b407-372915f0e09e None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:42.490449 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-6263816b-8ace-411d-b407-372915f0e09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:42.636612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 235/457] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:42.783238 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa233517-16ca-4f31-8e68-c0de55c3f280 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:42.783681 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa233517-16ca-4f31-8e68-c0de55c3f280 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:42.783781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa233517-16ca-4f31-8e68-c0de55c3f280 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:42.785661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 223/458] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:42.792166 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d553f1-539e-49de-b8ee-4715eea87a48 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:42.792536 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d553f1-539e-49de-b8ee-4715eea87a48 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:42.792767 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d553f1-539e-49de-b8ee-4715eea87a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:42.828060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-b3d553f1-539e-49de-b8ee-4715eea87a48 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:42.829363 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b3d553f1-539e-49de-b8ee-4715eea87a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:42.868981 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:42.948667 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156be95c-9f95-435a-9877-3e19801a6d49 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:42.948984 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156be95c-9f95-435a-9877-3e19801a6d49 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:42.949164 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156be95c-9f95-435a-9877-3e19801a6d49 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:42.950747 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 224/459] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:42.956895 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c394cff-493d-4989-aa64-e679a1951d87 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:42.957230 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c394cff-493d-4989-aa64-e679a1951d87 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:42.957504 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c394cff-493d-4989-aa64-e679a1951d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:42.991019 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 236/460] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 08:37:43.005106 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-4c394cff-493d-4989-aa64-e679a1951d87 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:43.006332 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4c394cff-493d-4989-aa64-e679a1951d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:43.152691 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 225/461] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:43.175315 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d553f1-539e-49de-b8ee-4715eea87a48 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:43.270752 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ae5c8-5531-457a-ac77-d5e3dceb3b80 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:43.271023 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ae5c8-5531-457a-ac77-d5e3dceb3b80 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:43.271257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ae5c8-5531-457a-ac77-d5e3dceb3b80 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:43.272768 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 226/462] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:43.278993 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95687370-9f6b-4ff6-b7fe-ca466824409b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:43.279279 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95687370-9f6b-4ff6-b7fe-ca466824409b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:43.279560 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95687370-9f6b-4ff6-b7fe-ca466824409b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:43.300561 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-95687370-9f6b-4ff6-b7fe-ca466824409b None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:43.301876 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-95687370-9f6b-4ff6-b7fe-ca466824409b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:43.322037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d553f1-539e-49de-b8ee-4715eea87a48 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:43.350114 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:43.386242 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 227/463] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 08:37:43.387228 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95687370-9f6b-4ff6-b7fe-ca466824409b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:43.387879 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c07eff-9490-4403-af7f-edf9820657b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:43.388742 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c07eff-9490-4403-af7f-edf9820657b6 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:43.389183 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c07eff-9490-4403-af7f-edf9820657b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:43.389587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c07eff-9490-4403-af7f-edf9820657b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:43.400795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c07eff-9490-4403-af7f-edf9820657b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:43.402499 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c07eff-9490-4403-af7f-edf9820657b6 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:43.432849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-88c07eff-9490-4403-af7f-edf9820657b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:43.432849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:43.432849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:43.432849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:43.432849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:43.432849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:43.432849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:43.432849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:43.432849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:43.432849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:43.432849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:43.432849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:43.432849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:37:43.432849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:37:43.432849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:37:43.432849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:37:43.433938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:43.433938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:43.433938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:43.433938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:43.433938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:43.433938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:43.433938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:43.433938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:43.433938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:43.433938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:43.433938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:43.433938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:43.433938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:43.433938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:43.433938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:43.433938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:43.433938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:37:43.434779 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:37:43.434779 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:43.434779 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:43.437828 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 237/464] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:43.446441 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c07eff-9490-4403-af7f-edf9820657b6 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:43.471253 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:43.563907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95687370-9f6b-4ff6-b7fe-ca466824409b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:43.599907 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c07eff-9490-4403-af7f-edf9820657b6 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:43.611877 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732bdf7c-b426-4e49-b58d-18c56cefcc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:43.612803 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732bdf7c-b426-4e49-b58d-18c56cefcc60 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:43.613093 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732bdf7c-b426-4e49-b58d-18c56cefcc60 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:43.613436 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732bdf7c-b426-4e49-b58d-18c56cefcc60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:43.628485 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732bdf7c-b426-4e49-b58d-18c56cefcc60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:43.630752 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732bdf7c-b426-4e49-b58d-18c56cefcc60 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:43.637250 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd99ae-09ab-4ff9-95d5-848d64b43f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:43.638121 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd99ae-09ab-4ff9-95d5-848d64b43f95 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:43.638493 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd99ae-09ab-4ff9-95d5-848d64b43f95 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:43.638866 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd99ae-09ab-4ff9-95d5-848d64b43f95 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:43.639652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cd99ae-09ab-4ff9-95d5-848d64b43f95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:43.640116 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cd99ae-09ab-4ff9-95d5-848d64b43f95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:43.641622 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cd99ae-09ab-4ff9-95d5-848d64b43f95 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:43.649043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-732bdf7c-b426-4e49-b58d-18c56cefcc60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:43.649043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:43.649043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:43.649043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:43.649043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:43.649043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:43.649043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:43.649043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:43.649043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:43.649043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:43.649043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:43.649043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:43.649043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:37:43.649043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:37:43.649043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:37:43.649043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:37:43.650127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:43.650127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:43.650127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:43.650127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:43.650127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:43.650127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:43.650127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:43.650127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:43.650127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:43.650127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:43.650127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:43.650127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:43.650127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:43.650127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:43.650127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:43.650127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:43.650127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:37:43.650998 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:37:43.650998 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:43.650998 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:43.650998 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 228/465] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:43.651336 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732bdf7c-b426-4e49-b58d-18c56cefcc60 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:43.665301 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 238/466] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:37:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:43.666298 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd99ae-09ab-4ff9-95d5-848d64b43f95 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:43.695318 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:43.744346 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:43.778389 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd99ae-09ab-4ff9-95d5-848d64b43f95 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:43.787637 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732bdf7c-b426-4e49-b58d-18c56cefcc60 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:43.820238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013c9f7-75f6-4820-8e16-0d5e6f689827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:43.821198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013c9f7-75f6-4820-8e16-0d5e6f689827 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:43.821674 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013c9f7-75f6-4820-8e16-0d5e6f689827 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:43.822122 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013c9f7-75f6-4820-8e16-0d5e6f689827 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:43.823000 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013c9f7-75f6-4820-8e16-0d5e6f689827 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:43.823499 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013c9f7-75f6-4820-8e16-0d5e6f689827 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:43.825086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013c9f7-75f6-4820-8e16-0d5e6f689827 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:43.830300 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4e469-c9d4-4780-b616-15286967c4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:43.831001 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4e469-c9d4-4780-b616-15286967c4db None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:43.831329 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4e469-c9d4-4780-b616-15286967c4db None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:43.831602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4e469-c9d4-4780-b616-15286967c4db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:43.837587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 239/467] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:37:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:43.842875 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013c9f7-75f6-4820-8e16-0d5e6f689827 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:43.844148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df4e469-c9d4-4780-b616-15286967c4db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:43.845602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df4e469-c9d4-4780-b616-15286967c4db None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:43.860868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-5df4e469-c9d4-4780-b616-15286967c4db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:43.860868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:43.860868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:43.860868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:43.860868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:43.860868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:43.860868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:43.860868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:43.860868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:43.860868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:43.860868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:43.860868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:43.860868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:37:43.860868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:37:43.860868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:37:43.860868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:37:43.862144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:43.862144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:43.862144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:43.862144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:43.862144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:43.862144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:43.862144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:43.862144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:43.862144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:43.862144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:43.862144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:43.862144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:43.862144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:43.862144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:43.862144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:43.862144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:43.862144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:37:43.863790 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:37:43.863790 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:43.863790 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:43.863790 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 229/468] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:43.863790 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4e469-c9d4-4780-b616-15286967c4db None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:43.952881 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:43.971616 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013c9f7-75f6-4820-8e16-0d5e6f689827 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:43.978483 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4e469-c9d4-4780-b616-15286967c4db None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:44.003293 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:44.011456 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79253e-d5bf-445a-860b-2f72d0989fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:44.012113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79253e-d5bf-445a-860b-2f72d0989fa9 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:44.012400 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79253e-d5bf-445a-860b-2f72d0989fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:44.012680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79253e-d5bf-445a-860b-2f72d0989fa9 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:44.013337 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac79253e-d5bf-445a-860b-2f72d0989fa9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:44.014792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac79253e-d5bf-445a-860b-2f72d0989fa9 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:44.016455 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c3352-a435-4844-a04c-999729e8c04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:44.017085 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c3352-a435-4844-a04c-999729e8c04d None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:44.017362 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c3352-a435-4844-a04c-999729e8c04d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:44.017633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c3352-a435-4844-a04c-999729e8c04d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:44.030782 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c3352-a435-4844-a04c-999729e8c04d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:44.032452 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c3352-a435-4844-a04c-999729e8c04d None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:44.046079 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 240/469] 10.222.0.71 () {60 vars in 1132 bytes} [Fri Aug 26 08:37:43 2022] POST /identity/v3/users => generated 246 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:37:44.046079 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79253e-d5bf-445a-860b-2f72d0989fa9 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:44.047381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-265c3352-a435-4844-a04c-999729e8c04d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:44.047381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:44.047381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:44.047381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:44.047381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:44.047381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:44.047381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:44.047381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:44.047381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:44.047381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:44.047381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:44.047381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:44.047381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:37:44.047381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:37:44.047381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:37:44.047381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:37:44.048257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:44.048257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:44.048257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:44.048257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:44.048257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:44.048257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:44.048257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:44.048257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:44.048257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:44.048257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:44.048257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:44.048257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:44.048257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:44.048257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:44.048257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:44.048257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:44.048257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:37:44.049109 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:37:44.049109 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:44.049109 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:44.049109 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 230/470] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:44.049767 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c3352-a435-4844-a04c-999729e8c04d None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:44.176999 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c3352-a435-4844-a04c-999729e8c04d None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:44.188511 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac79253e-d5bf-445a-860b-2f72d0989fa9 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:44.211922 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:44.221517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d28805-e4f8-4163-a786-5148e78d71aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:44.222131 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d28805-e4f8-4163-a786-5148e78d71aa None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:44.222414 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d28805-e4f8-4163-a786-5148e78d71aa None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:44.222718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d28805-e4f8-4163-a786-5148e78d71aa None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:44.223433 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d28805-e4f8-4163-a786-5148e78d71aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:44.223740 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d28805-e4f8-4163-a786-5148e78d71aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:44.225128 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d28805-e4f8-4163-a786-5148e78d71aa None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:44.231286 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87fe0b-691f-42b0-8122-b7d1b0b09803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:44.231966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87fe0b-691f-42b0-8122-b7d1b0b09803 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:44.232263 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87fe0b-691f-42b0-8122-b7d1b0b09803 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:44.232522 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87fe0b-691f-42b0-8122-b7d1b0b09803 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:44.233172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b87fe0b-691f-42b0-8122-b7d1b0b09803 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:44.234625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b87fe0b-691f-42b0-8122-b7d1b0b09803 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:44.242179 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 231/471] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:37:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:44.243100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d28805-e4f8-4163-a786-5148e78d71aa None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:44.266131 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 241/472] 10.222.0.71 () {60 vars in 1132 bytes} [Fri Aug 26 08:37:44 2022] POST /identity/v3/users => generated 248 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:37:44.267987 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fcf0e-d748-4002-bd5c-f994002ab06f None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:44.268472 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fcf0e-d748-4002-bd5c-f994002ab06f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:44.268961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fcf0e-d748-4002-bd5c-f994002ab06f None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:44.275086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 242/473] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:44.276433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29fcf0e-d748-4002-bd5c-f994002ab06f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:44.338713 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:44.375317 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d28805-e4f8-4163-a786-5148e78d71aa None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:44.416907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f083b42-22bd-4284-9cff-772429f5956c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:44.417871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f083b42-22bd-4284-9cff-772429f5956c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:44.418329 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f083b42-22bd-4284-9cff-772429f5956c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:44.418776 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f083b42-22bd-4284-9cff-772429f5956c None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:44.419654 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f083b42-22bd-4284-9cff-772429f5956c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:44.420168 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f083b42-22bd-4284-9cff-772429f5956c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:44.421857 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f083b42-22bd-4284-9cff-772429f5956c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:44.426594 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29fcf0e-d748-4002-bd5c-f994002ab06f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:44.435676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 232/474] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:37:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:44.441773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0556e-3bf7-4800-8ded-974f8b8fd7a2 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:44.442445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0556e-3bf7-4800-8ded-974f8b8fd7a2 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:44.442982 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0556e-3bf7-4800-8ded-974f8b8fd7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:44.464050 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-7aa0556e-3bf7-4800-8ded-974f8b8fd7a2 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:44.465618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7aa0556e-3bf7-4800-8ded-974f8b8fd7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:44.469362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c643bb-47fb-4458-a750-e2374f49dba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:44.470022 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c643bb-47fb-4458-a750-e2374f49dba1 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:44.470297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c643bb-47fb-4458-a750-e2374f49dba1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:44.470564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c643bb-47fb-4458-a750-e2374f49dba1 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:44.471242 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c643bb-47fb-4458-a750-e2374f49dba1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:44.472806 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c643bb-47fb-4458-a750-e2374f49dba1 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:44.497442 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 243/475] 10.222.0.71 () {60 vars in 1132 bytes} [Fri Aug 26 08:37:44 2022] POST /identity/v3/users => generated 248 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:37:44.498624 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c643bb-47fb-4458-a750-e2374f49dba1 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:44.544981 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:44.589789 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 233/476] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:44.658834 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:44.663844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c643bb-47fb-4458-a750-e2374f49dba1 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:44.704483 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ac491-fb4c-4e0c-a663-efd0d3ec6fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:44.704998 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ac491-fb4c-4e0c-a663-efd0d3ec6fe6 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:44.705277 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ac491-fb4c-4e0c-a663-efd0d3ec6fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:44.705549 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ac491-fb4c-4e0c-a663-efd0d3ec6fe6 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:44.706196 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38ac491-fb4c-4e0c-a663-efd0d3ec6fe6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:44.707747 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38ac491-fb4c-4e0c-a663-efd0d3ec6fe6 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:44.731374 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 244/477] 10.222.0.71 () {60 vars in 1132 bytes} [Fri Aug 26 08:37:44 2022] POST /identity/v3/users => generated 249 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:37:44.869167 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa0556e-3bf7-4800-8ded-974f8b8fd7a2 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:44.981001 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa0556e-3bf7-4800-8ded-974f8b8fd7a2 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:45.022466 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fe6cef-2552-401a-99c7-6f7754fe80bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:45.023115 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe6cef-2552-401a-99c7-6f7754fe80bc None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:45.023394 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe6cef-2552-401a-99c7-6f7754fe80bc None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:45.023662 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe6cef-2552-401a-99c7-6f7754fe80bc None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:45.024347 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fe6cef-2552-401a-99c7-6f7754fe80bc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:45.025727 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fe6cef-2552-401a-99c7-6f7754fe80bc None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:45.034512 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.sql.core [None req-70fe6cef-2552-401a-99c7-6f7754fe80bc None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 26 08:37:45.034512 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 26 08:37:45.034512 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [parameters: {'id': '1c35129713e64c5899ddeb3d7e8f18b1', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 26 08:37:45.034512 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93173) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 26 08:37:45.053892 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-70fe6cef-2552-401a-99c7-6f7754fe80bc None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 26 08:37:45.053892 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:45.053892 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 08:37:45.053892 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 08:37:45.053892 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 08:37:45.053892 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 08:37:45.053892 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 08:37:45.053892 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 08:37:45.053892 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 08:37:45.053892 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application conn.query(q) Aug 26 08:37:45.053892 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 08:37:45.053892 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 08:37:45.053892 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 08:37:45.053892 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application result.read() Aug 26 08:37:45.053892 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 08:37:45.054798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 08:37:45.054798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 08:37:45.054798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 08:37:45.054798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 08:37:45.054798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 08:37:45.054798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 08:37:45.054798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 08:37:45.054798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 26 08:37:45.054798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:45.054798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 26 08:37:45.054798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:45.054798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:45.054798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 26 08:37:45.054798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 26 08:37:45.054798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 26 08:37:45.054798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return ref.to_dict() Aug 26 08:37:45.054798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 08:37:45.054798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application next(self.gen) Aug 26 08:37:45.055713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 26 08:37:45.055713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application yield resource Aug 26 08:37:45.055713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 08:37:45.055713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application next(self.gen) Aug 26 08:37:45.055713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 26 08:37:45.055713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self.session.rollback() Aug 26 08:37:45.055713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 26 08:37:45.055713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self.force_reraise() Aug 26 08:37:45.055713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 26 08:37:45.055713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise self.value Aug 26 08:37:45.055713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 26 08:37:45.055713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 26 08:37:45.055713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 26 08:37:45.055713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application session.commit() Aug 26 08:37:45.055713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 26 08:37:45.055713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 26 08:37:45.055713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 26 08:37:45.056618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._prepare_impl() Aug 26 08:37:45.056618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 26 08:37:45.056618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self.session.flush() Aug 26 08:37:45.056618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 26 08:37:45.056618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._flush(objects) Aug 26 08:37:45.056618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 26 08:37:45.056618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 26 08:37:45.056618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 26 08:37:45.056618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application compat.raise_( Aug 26 08:37:45.056618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 08:37:45.056618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception Aug 26 08:37:45.056618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 26 08:37:45.056618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application flush_context.execute() Aug 26 08:37:45.056618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 26 08:37:45.056618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rec.execute(self) Aug 26 08:37:45.056618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 26 08:37:45.056618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 26 08:37:45.057583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 26 08:37:45.057583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 26 08:37:45.057583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 26 08:37:45.057583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 26 08:37:45.057583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 26 08:37:45.057583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 26 08:37:45.057583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 26 08:37:45.057583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 26 08:37:45.057583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 26 08:37:45.057583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 26 08:37:45.057583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 26 08:37:45.057583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 26 08:37:45.057583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 26 08:37:45.057583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 26 08:37:45.057583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 08:37:45.057583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception Aug 26 08:37:45.058406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 08:37:45.058406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 08:37:45.058406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 08:37:45.058406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 08:37:45.058406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 08:37:45.058406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 08:37:45.058406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 08:37:45.058406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application conn.query(q) Aug 26 08:37:45.058406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 08:37:45.058406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 08:37:45.058406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 08:37:45.058406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application result.read() Aug 26 08:37:45.058406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 08:37:45.058406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 08:37:45.058406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 08:37:45.058406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 08:37:45.060531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 08:37:45.060531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 08:37:45.060531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 08:37:45.060531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 08:37:45.060531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 26 08:37:45.060531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 26 08:37:45.060531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [parameters: {'id': '1c35129713e64c5899ddeb3d7e8f18b1', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 26 08:37:45.060531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 26 08:37:45.060531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:45.060531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:37:45.060531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:45.060531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:45.060531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:45.060531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:45.060531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:45.060531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:45.062107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:45.062107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:45.062107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:45.062107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:45.062107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:45.062107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:45.062107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 26 08:37:45.062107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 26 08:37:45.062107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:45.062107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:45.062107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 26 08:37:45.062107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 26 08:37:45.062107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 26 08:37:45.062107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 26 08:37:45.062107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 26 08:37:45.062107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:45.064023 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 234/478] 10.222.0.71 () {60 vars in 1131 bytes} [Fri Aug 26 08:37:44 2022] POST /identity/v3/roles => generated 144 bytes in 187 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:37:45.064162 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ac491-fb4c-4e0c-a663-efd0d3ec6fe6 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:45.179224 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ac491-fb4c-4e0c-a663-efd0d3ec6fe6 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:45.217015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a854f-87d0-48a5-9f59-028038f30603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:45.217670 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a854f-87d0-48a5-9f59-028038f30603 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:45.217944 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a854f-87d0-48a5-9f59-028038f30603 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:45.218217 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a854f-87d0-48a5-9f59-028038f30603 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:45.229652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a854f-87d0-48a5-9f59-028038f30603 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:45.235393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-683a854f-87d0-48a5-9f59-028038f30603 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 08:37:45.235393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:45.235393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:45.235393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:45.235393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:45.235393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:45.235393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:45.235393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:45.235393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:45.235393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:45.235393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:45.235393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:45.235393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:45.235393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:45.235393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:45.235393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:45.237508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:45.237508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:45.237508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:45.237508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:45.237508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:45.237508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:45.237508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:45.237508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:45.237508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:45.237508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:45.237508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:45.237508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:45.237508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:45.237508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:45.237508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:45.237508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:45.238746 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:45.238746 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:45.238746 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:45.238746 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:45.238746 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:45.238746 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:45.238746 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:45.238746 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:45.238746 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:45.238746 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:45.238746 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 08:37:45.238746 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:45.238746 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 245/479] 10.222.0.71 () {56 vars in 1114 bytes} [Fri Aug 26 08:37:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:45.243245 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fe6cef-2552-401a-99c7-6f7754fe80bc None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:45.291544 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:45.366055 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fe6cef-2552-401a-99c7-6f7754fe80bc None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:45.408329 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427836bc-1372-4cb8-9e1d-827d540c49b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:45.408955 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427836bc-1372-4cb8-9e1d-827d540c49b6 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:45.409213 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427836bc-1372-4cb8-9e1d-827d540c49b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:45.409483 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427836bc-1372-4cb8-9e1d-827d540c49b6 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:45.410160 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427836bc-1372-4cb8-9e1d-827d540c49b6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:45.410492 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427836bc-1372-4cb8-9e1d-827d540c49b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:45.411897 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427836bc-1372-4cb8-9e1d-827d540c49b6 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:45.421024 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 235/480] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:37:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:37:46.252018 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde2b68-d4ac-45e1-a359-0bd285543773 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:46.252810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde2b68-d4ac-45e1-a359-0bd285543773 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:46.252810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde2b68-d4ac-45e1-a359-0bd285543773 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:46.254252 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 246/481] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:46.260296 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07340d0a-b87e-4838-b33b-f94d8a258f9e None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:46.260904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07340d0a-b87e-4838-b33b-f94d8a258f9e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:46.260904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07340d0a-b87e-4838-b33b-f94d8a258f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:46.284505 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-07340d0a-b87e-4838-b33b-f94d8a258f9e None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:46.285889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-07340d0a-b87e-4838-b33b-f94d8a258f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:46.380050 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 236/482] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 08:37:46.720305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5bff5-e1d8-4ce8-bd43-264fc9d72a49 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:46.720580 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5bff5-e1d8-4ce8-bd43-264fc9d72a49 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:46.720851 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5bff5-e1d8-4ce8-bd43-264fc9d72a49 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:46.722473 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 247/483] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:46.727987 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dd60d-e516-44fc-885b-8c26174efc5f None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:46.728274 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dd60d-e516-44fc-885b-8c26174efc5f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:46.728546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dd60d-e516-44fc-885b-8c26174efc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:46.752550 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-512dd60d-e516-44fc-885b-8c26174efc5f None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:46.753754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-512dd60d-e516-44fc-885b-8c26174efc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:46.787810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5bff5-e1d8-4ce8-bd43-264fc9d72a49 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:46.808623 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:46.811265 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:46.905842 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 237/484] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:46.994493 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5bff5-e1d8-4ce8-bd43-264fc9d72a49 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:47.039348 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b4245-c9c9-422e-bcb1-f5fe19ed30ba None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:47.039678 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b4245-c9c9-422e-bcb1-f5fe19ed30ba None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:47.039888 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b4245-c9c9-422e-bcb1-f5fe19ed30ba None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:47.041279 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 238/485] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:37:47.049299 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db052baf-ff67-4bf7-b8aa-1020e2dc95dd None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:47.049799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db052baf-ff67-4bf7-b8aa-1020e2dc95dd None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:47.050272 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db052baf-ff67-4bf7-b8aa-1020e2dc95dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:47.059011 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d780ff5-37ba-4024-b84d-d809086efdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:47.059768 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d780ff5-37ba-4024-b84d-d809086efdec None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:47.060059 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d780ff5-37ba-4024-b84d-d809086efdec None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:47.060335 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d780ff5-37ba-4024-b84d-d809086efdec None admin] PATH_INFO: `/v3/roles/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:47.073079 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d780ff5-37ba-4024-b84d-d809086efdec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:47.078050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-9d780ff5-37ba-4024-b84d-d809086efdec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:37:47.078050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:47.078050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:47.078050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:47.078050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:47.078050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:47.078050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:47.078050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:47.078050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:47.078050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:47.078050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:47.078050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:47.078050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:47.078050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:47.078050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:47.078050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:47.079008 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:47.079008 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:47.079008 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:47.079008 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:47.079008 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:47.079008 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:47.079008 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:47.079008 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:47.079008 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:47.079008 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:47.079008 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:47.079008 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:47.079008 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:47.079008 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:47.079008 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:47.079008 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:47.079803 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:47.079803 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:47.079803 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:47.079803 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:47.079803 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:47.079803 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:47.079803 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:47.079803 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:47.079803 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:47.079803 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:47.079803 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:37:47.079803 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:47.079803 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 248/486] 10.222.0.71 () {56 vars in 1102 bytes} [Fri Aug 26 08:37:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:47.085848 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d780ff5-37ba-4024-b84d-d809086efdec None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:47.087152 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-db052baf-ff67-4bf7-b8aa-1020e2dc95dd None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:47.088989 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-db052baf-ff67-4bf7-b8aa-1020e2dc95dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:47.211069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 239/487] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:47.212106 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db052baf-ff67-4bf7-b8aa-1020e2dc95dd None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:47.233450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:47.264184 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d780ff5-37ba-4024-b84d-d809086efdec None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:47.343868 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146cb58-39b4-4bf5-912f-74fd479b1234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:47.344545 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146cb58-39b4-4bf5-912f-74fd479b1234 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:47.344843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146cb58-39b4-4bf5-912f-74fd479b1234 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:47.345180 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146cb58-39b4-4bf5-912f-74fd479b1234 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:47.345981 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146cb58-39b4-4bf5-912f-74fd479b1234 None admin] RBAC: Adding query filter params (name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:47.346469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146cb58-39b4-4bf5-912f-74fd479b1234 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:47.348038 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146cb58-39b4-4bf5-912f-74fd479b1234 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:47.366387 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 249/488] 10.222.0.71 () {56 vars in 1103 bytes} [Fri Aug 26 08:37:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:47.381670 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832b1c4-64e1-4019-965b-82dc77b9ef3d None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:47.382331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832b1c4-64e1-4019-965b-82dc77b9ef3d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:47.382613 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832b1c4-64e1-4019-965b-82dc77b9ef3d None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:47.384568 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 250/489] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:37:47.384684 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:47.385457 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5832b1c4-64e1-4019-965b-82dc77b9ef3d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:47.396053 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db052baf-ff67-4bf7-b8aa-1020e2dc95dd None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:47.434696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8862f240-3ef0-4e26-97a2-e553c0d7170d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:47.435383 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862f240-3ef0-4e26-97a2-e553c0d7170d None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:47.435696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862f240-3ef0-4e26-97a2-e553c0d7170d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:47.435919 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862f240-3ef0-4e26-97a2-e553c0d7170d None admin] PATH_INFO: `/v3/roles/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:47.446938 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8862f240-3ef0-4e26-97a2-e553c0d7170d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:47.451603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-8862f240-3ef0-4e26-97a2-e553c0d7170d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:37:47.451603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:47.451603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:47.451603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:47.451603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:47.451603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:47.451603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:47.451603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:47.451603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:47.451603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:47.451603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:47.451603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:47.451603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:47.451603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:47.451603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:47.451603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:47.452479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:47.452479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:47.452479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:47.452479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:47.452479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:47.452479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:47.452479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:47.452479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:47.452479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:47.452479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:47.452479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:47.452479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:47.452479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:47.452479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:47.452479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:47.452479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:47.453268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:47.453268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:47.453268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:47.453268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:47.453268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:47.453268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:47.453268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:47.453268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:47.453268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:47.453268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:47.453268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:37:47.453268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:47.453268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 240/490] 10.222.0.71 () {56 vars in 1102 bytes} [Fri Aug 26 08:37:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:47.453970 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561beae9-97d0-463a-a96a-14ea81b893be None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:47.454244 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561beae9-97d0-463a-a96a-14ea81b893be None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:47.454543 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561beae9-97d0-463a-a96a-14ea81b893be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:47.474084 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-561beae9-97d0-463a-a96a-14ea81b893be None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:47.475352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-561beae9-97d0-463a-a96a-14ea81b893be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:47.531114 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5832b1c4-64e1-4019-965b-82dc77b9ef3d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:47.549990 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 241/491] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:47.550790 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561beae9-97d0-463a-a96a-14ea81b893be None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:47.581468 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5dc24-287e-4e4f-88f3-6d106ac83d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:47.582179 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5dc24-287e-4e4f-88f3-6d106ac83d53 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:47.582529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5dc24-287e-4e4f-88f3-6d106ac83d53 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:47.582862 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5dc24-287e-4e4f-88f3-6d106ac83d53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:47.586199 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:47.595069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b5dc24-287e-4e4f-88f3-6d106ac83d53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:47.596733 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b5dc24-287e-4e4f-88f3-6d106ac83d53 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:47.612705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-c6b5dc24-287e-4e4f-88f3-6d106ac83d53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:47.612705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:47.612705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:47.612705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:47.612705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:47.612705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:47.612705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:47.612705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:47.612705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:47.612705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:47.612705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:47.612705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:47.612705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:37:47.612705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:37:47.612705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:37:47.612705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:37:47.613737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:47.613737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:47.613737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:47.613737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:47.613737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:47.613737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:47.613737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:47.613737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:47.613737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:47.613737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:47.613737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:47.613737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:47.613737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:47.613737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:47.613737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:47.613737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:47.613737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:37:47.614580 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:37:47.614580 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:47.614580 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:47.614580 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 251/492] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:47.615811 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5dc24-287e-4e4f-88f3-6d106ac83d53 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:47.671449 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:47.697176 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561beae9-97d0-463a-a96a-14ea81b893be None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:47.729537 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5dc24-287e-4e4f-88f3-6d106ac83d53 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:47.743710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2f19b-5f0d-4fda-b6e1-fa20788ddbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:47.744590 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2f19b-5f0d-4fda-b6e1-fa20788ddbe4 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:47.744878 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2f19b-5f0d-4fda-b6e1-fa20788ddbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:47.745148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2f19b-5f0d-4fda-b6e1-fa20788ddbe4 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:47.746350 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2f19b-5f0d-4fda-b6e1-fa20788ddbe4 None admin] RBAC: Adding query filter params (name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:47.746660 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2f19b-5f0d-4fda-b6e1-fa20788ddbe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:47.748600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e2f19b-5f0d-4fda-b6e1-fa20788ddbe4 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:47.761574 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 242/493] 10.222.0.71 () {56 vars in 1103 bytes} [Fri Aug 26 08:37:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:47.762677 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2f19b-5f0d-4fda-b6e1-fa20788ddbe4 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:47.773343 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136efa69-5166-4c71-a526-ff7f25b71bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:47.775302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136efa69-5166-4c71-a526-ff7f25b71bcb None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:47.775973 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136efa69-5166-4c71-a526-ff7f25b71bcb None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:47.777012 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136efa69-5166-4c71-a526-ff7f25b71bcb None admin] PATH_INFO: `/v3/roles/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:47.801293 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136efa69-5166-4c71-a526-ff7f25b71bcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:47.809816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-136efa69-5166-4c71-a526-ff7f25b71bcb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:37:47.809816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:47.809816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:47.809816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:47.809816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:47.809816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:47.809816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:47.809816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:47.809816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:47.809816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:47.809816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:47.809816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:47.809816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:47.809816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:47.809816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:47.809816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:47.811702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:47.811702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:47.811702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:47.811702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:47.811702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:47.811702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:47.811702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:47.811702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:47.811702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:47.811702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:47.811702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:47.811702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:47.811702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:47.811702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:47.811702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:47.811702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:47.813467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:47.813467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:47.813467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:47.813467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:47.813467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:47.813467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:47.813467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:47.813467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:47.813467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:47.813467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:47.813467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:37:47.813467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:47.818936 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 252/494] 10.222.0.71 () {56 vars in 1102 bytes} [Fri Aug 26 08:37:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:47.818936 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136efa69-5166-4c71-a526-ff7f25b71bcb None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:47.883984 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e2f19b-5f0d-4fda-b6e1-fa20788ddbe4 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:47.884385 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:47.935338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb2396-10c8-45de-bd8b-7f4470a35844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:47.936507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb2396-10c8-45de-bd8b-7f4470a35844 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:47.937365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb2396-10c8-45de-bd8b-7f4470a35844 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:47.938056 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb2396-10c8-45de-bd8b-7f4470a35844 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:47.938937 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb2396-10c8-45de-bd8b-7f4470a35844 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:47.939719 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb2396-10c8-45de-bd8b-7f4470a35844 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:47.942291 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb2396-10c8-45de-bd8b-7f4470a35844 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:47.947273 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136efa69-5166-4c71-a526-ff7f25b71bcb None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:47.960173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 243/495] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:47.961617 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb2396-10c8-45de-bd8b-7f4470a35844 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:47.974500 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:47.985939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d34278-b1ec-4904-88ec-23e25ef397bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:47.986577 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d34278-b1ec-4904-88ec-23e25ef397bf None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:47.986857 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d34278-b1ec-4904-88ec-23e25ef397bf None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:47.987164 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d34278-b1ec-4904-88ec-23e25ef397bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:47.998543 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d34278-b1ec-4904-88ec-23e25ef397bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:48.000062 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d34278-b1ec-4904-88ec-23e25ef397bf None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:48.015089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-c1d34278-b1ec-4904-88ec-23e25ef397bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:48.015089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:48.015089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:48.015089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:48.015089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:48.015089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:48.015089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:48.015089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:48.015089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:48.015089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:48.015089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:48.015089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:48.015089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:37:48.015089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:37:48.015089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:37:48.015089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:37:48.016073 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:48.016073 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:48.016073 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:48.016073 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:48.016073 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:48.016073 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:48.016073 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:48.016073 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:48.016073 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:48.016073 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:48.016073 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:48.016073 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:48.016073 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:48.016073 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:48.016073 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:48.016073 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:48.016073 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:37:48.016890 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:37:48.016890 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:48.016890 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:48.016890 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 253/496] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:48.017751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d34278-b1ec-4904-88ec-23e25ef397bf None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:48.088218 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb2396-10c8-45de-bd8b-7f4470a35844 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:48.138704 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe13c0-6082-4964-b87a-b59cf04cbb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:48.139421 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe13c0-6082-4964-b87a-b59cf04cbb23 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:48.139764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe13c0-6082-4964-b87a-b59cf04cbb23 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:48.140073 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe13c0-6082-4964-b87a-b59cf04cbb23 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:48.141152 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe13c0-6082-4964-b87a-b59cf04cbb23 None admin] RBAC: Adding query filter params (name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:48.141152 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe13c0-6082-4964-b87a-b59cf04cbb23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:48.142922 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe13c0-6082-4964-b87a-b59cf04cbb23 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:48.146483 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:48.152504 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d34278-b1ec-4904-88ec-23e25ef397bf None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:48.159484 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 244/497] 10.222.0.71 () {56 vars in 1103 bytes} [Fri Aug 26 08:37:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:48.160797 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe13c0-6082-4964-b87a-b59cf04cbb23 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:48.202728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a297781-04b2-482c-9608-2c6ddbdfc96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:48.203963 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a297781-04b2-482c-9608-2c6ddbdfc96a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:48.204405 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a297781-04b2-482c-9608-2c6ddbdfc96a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:48.204846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a297781-04b2-482c-9608-2c6ddbdfc96a None admin] PATH_INFO: `/v3/roles/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:48.222325 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a297781-04b2-482c-9608-2c6ddbdfc96a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:48.227884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-3a297781-04b2-482c-9608-2c6ddbdfc96a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:37:48.227884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:48.227884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:48.227884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:48.227884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:48.227884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:48.227884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:48.227884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:48.227884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:48.227884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:48.227884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:48.227884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:48.227884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:48.227884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:48.227884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:48.227884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:48.229072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:48.229072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:48.229072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:48.229072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:48.229072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:48.229072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:48.229072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:48.229072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:48.229072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:48.229072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:48.229072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:48.229072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:48.229072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:48.229072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:48.229072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:48.229072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:48.230384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:48.230384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:48.230384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:48.230384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:48.230384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:48.230384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:48.230384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:48.230384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:48.230384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:48.230384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:48.230384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:37:48.230384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:48.230384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 254/498] 10.222.0.71 () {56 vars in 1102 bytes} [Fri Aug 26 08:37:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:48.230384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a297781-04b2-482c-9608-2c6ddbdfc96a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:48.265571 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:48.314993 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe13c0-6082-4964-b87a-b59cf04cbb23 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:48.351910 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a297781-04b2-482c-9608-2c6ddbdfc96a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:48.353896 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e8f90-a84a-4ba2-98fe-2bc9ed714bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:48.354707 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e8f90-a84a-4ba2-98fe-2bc9ed714bbd None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:48.355016 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e8f90-a84a-4ba2-98fe-2bc9ed714bbd None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:48.355328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e8f90-a84a-4ba2-98fe-2bc9ed714bbd None admin] PATH_INFO: `/v3/users/nova` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:48.363749 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e8f90-a84a-4ba2-98fe-2bc9ed714bbd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:48.365244 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e8f90-a84a-4ba2-98fe-2bc9ed714bbd None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:48.378386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-5d6e8f90-a84a-4ba2-98fe-2bc9ed714bbd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 08:37:48.378386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:48.378386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:48.378386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:48.378386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:48.378386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:48.378386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:48.378386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:48.378386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:48.378386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:48.378386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:48.378386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:48.378386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:37:48.378386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:37:48.378386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:37:48.378386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:37:48.379365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:48.379365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:48.379365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:37:48.379365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:37:48.379365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:37:48.379365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:37:48.379365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:48.379365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:48.379365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:48.379365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:48.379365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:48.379365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:48.379365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:48.379365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:48.379365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:48.379365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:48.379365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:48.379365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:48.380481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:48.380481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:48.380481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:37:48.380481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:37:48.380481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:37:48.380481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:37:48.380481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:37:48.380481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:37:48.380481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 08:37:48.380481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:48.380481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 245/499] 10.222.0.71 () {56 vars in 1093 bytes} [Fri Aug 26 08:37:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:48.381231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b7647-5511-415e-98c9-880b5f52c662 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:48.381231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b7647-5511-415e-98c9-880b5f52c662 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:48.381231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b7647-5511-415e-98c9-880b5f52c662 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:48.382618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 246/500] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:48.383182 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b7647-5511-415e-98c9-880b5f52c662 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:48.395428 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3563f-0726-46a2-a45d-9d52a64a1a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:48.396043 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3563f-0726-46a2-a45d-9d52a64a1a84 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:48.396292 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3563f-0726-46a2-a45d-9d52a64a1a84 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:48.396562 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c3563f-0726-46a2-a45d-9d52a64a1a84 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:48.397218 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3563f-0726-46a2-a45d-9d52a64a1a84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:48.397515 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3563f-0726-46a2-a45d-9d52a64a1a84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:48.398971 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c3563f-0726-46a2-a45d-9d52a64a1a84 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:48.411042 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 255/501] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:48.411883 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3563f-0726-46a2-a45d-9d52a64a1a84 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:48.434503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:48.500297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b7647-5511-415e-98c9-880b5f52c662 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:48.521367 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c3563f-0726-46a2-a45d-9d52a64a1a84 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:48.528186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:48.536979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc74596-6107-44f4-8b88-0cfa4581dd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:48.537603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc74596-6107-44f4-8b88-0cfa4581dd85 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:48.537872 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc74596-6107-44f4-8b88-0cfa4581dd85 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:48.538153 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc74596-6107-44f4-8b88-0cfa4581dd85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:48.549257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc74596-6107-44f4-8b88-0cfa4581dd85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:48.550697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc74596-6107-44f4-8b88-0cfa4581dd85 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:48.556607 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9ec1f-844d-46e3-ba6d-479bf4befe2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:48.557226 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9ec1f-844d-46e3-ba6d-479bf4befe2b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:48.557506 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9ec1f-844d-46e3-ba6d-479bf4befe2b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:48.557780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9ec1f-844d-46e3-ba6d-479bf4befe2b None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:48.558453 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9ec1f-844d-46e3-ba6d-479bf4befe2b None admin] RBAC: Adding query filter params (name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:48.559411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9ec1f-844d-46e3-ba6d-479bf4befe2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:48.560247 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9ec1f-844d-46e3-ba6d-479bf4befe2b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:48.565429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-7bc74596-6107-44f4-8b88-0cfa4581dd85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:48.565429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:48.565429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:48.565429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:48.565429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:48.565429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:48.565429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:48.565429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:48.565429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:48.565429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:48.565429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:48.565429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:48.565429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:37:48.565429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:37:48.565429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:37:48.565429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:37:48.566385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:48.566385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:48.566385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:48.566385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:48.566385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:48.566385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:48.566385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:48.566385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:48.566385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:48.566385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:48.566385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:48.566385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:48.566385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:48.566385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:48.566385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:48.566385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:48.566385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:37:48.567312 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:37:48.567312 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:48.567312 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:48.567312 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 247/502] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:48.567523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc74596-6107-44f4-8b88-0cfa4581dd85 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:48.574181 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 256/503] 10.222.0.71 () {56 vars in 1103 bytes} [Fri Aug 26 08:37:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:48.576683 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd6c3b-0650-450c-a6fa-596a49210b8f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:48.576683 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd6c3b-0650-450c-a6fa-596a49210b8f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:48.576683 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd6c3b-0650-450c-a6fa-596a49210b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:48.595359 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-54dd6c3b-0650-450c-a6fa-596a49210b8f None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:48.596590 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-54dd6c3b-0650-450c-a6fa-596a49210b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:48.667450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 257/504] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:48.668455 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd6c3b-0650-450c-a6fa-596a49210b8f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:48.671323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc74596-6107-44f4-8b88-0cfa4581dd85 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:48.691317 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:48.710453 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06050464-641b-4a43-9f8e-6c1027026aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:48.711109 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06050464-641b-4a43-9f8e-6c1027026aab None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:48.711392 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06050464-641b-4a43-9f8e-6c1027026aab None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:48.711668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06050464-641b-4a43-9f8e-6c1027026aab None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:48.712513 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06050464-641b-4a43-9f8e-6c1027026aab None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:48.712736 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06050464-641b-4a43-9f8e-6c1027026aab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:48.714238 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06050464-641b-4a43-9f8e-6c1027026aab None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:48.756861 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 248/505] 10.222.0.71 () {56 vars in 1133 bytes} [Fri Aug 26 08:37:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:48.757751 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06050464-641b-4a43-9f8e-6c1027026aab None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:48.788723 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd6c3b-0650-450c-a6fa-596a49210b8f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:48.802691 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:48.830459 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04206943-9ab7-46e7-b205-556fe79c33a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:48.831134 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04206943-9ab7-46e7-b205-556fe79c33a6 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:48.831476 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04206943-9ab7-46e7-b205-556fe79c33a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:48.831775 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04206943-9ab7-46e7-b205-556fe79c33a6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:48.840542 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04206943-9ab7-46e7-b205-556fe79c33a6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:48.842085 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04206943-9ab7-46e7-b205-556fe79c33a6 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:48.855579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-04206943-9ab7-46e7-b205-556fe79c33a6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 26 08:37:48.855579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:48.855579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:48.855579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:48.855579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:48.855579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:48.855579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:48.855579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:48.855579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:48.855579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:48.855579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:48.855579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:48.855579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:37:48.855579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:37:48.855579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:37:48.855579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:37:48.856547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:48.856547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:48.856547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:37:48.856547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:37:48.856547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:37:48.856547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:37:48.856547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:48.856547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:48.856547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:48.856547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:48.856547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:48.856547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:48.856547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:48.856547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:48.856547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:48.856547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:48.856547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:48.856547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:48.857427 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:48.857427 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:48.857427 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:37:48.857427 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:37:48.857427 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:37:48.857427 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:37:48.857427 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:37:48.857427 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:37:48.857427 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 26 08:37:48.857427 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:48.857427 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 258/506] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:37:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:48.857974 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04206943-9ab7-46e7-b205-556fe79c33a6 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:48.877921 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06050464-641b-4a43-9f8e-6c1027026aab None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:48.920632 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726c21d-d772-4d0a-89f5-2f3fe085aa66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:48.921128 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726c21d-d772-4d0a-89f5-2f3fe085aa66 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:48.921407 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726c21d-d772-4d0a-89f5-2f3fe085aa66 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:48.921679 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726c21d-d772-4d0a-89f5-2f3fe085aa66 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:48.922366 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4726c21d-d772-4d0a-89f5-2f3fe085aa66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:48.922640 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4726c21d-d772-4d0a-89f5-2f3fe085aa66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:48.924145 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4726c21d-d772-4d0a-89f5-2f3fe085aa66 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:48.938068 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 249/507] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:48.939364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726c21d-d772-4d0a-89f5-2f3fe085aa66 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:48.943249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:48.980788 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04206943-9ab7-46e7-b205-556fe79c33a6 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:49.024719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e9727-6d5e-4bac-9762-500fd24139b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:49.025474 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e9727-6d5e-4bac-9762-500fd24139b3 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:49.025820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e9727-6d5e-4bac-9762-500fd24139b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:49.026160 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e9727-6d5e-4bac-9762-500fd24139b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:49.037594 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e9727-6d5e-4bac-9762-500fd24139b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:49.039110 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e9727-6d5e-4bac-9762-500fd24139b3 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:49.052436 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:49.056569 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726c21d-d772-4d0a-89f5-2f3fe085aa66 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:49.066115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-467e9727-6d5e-4bac-9762-500fd24139b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:49.066115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:49.066115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:49.066115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:49.066115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:49.066115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:49.066115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:49.066115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:49.066115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:49.066115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:49.066115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:49.066115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:49.066115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:37:49.066115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:37:49.066115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:37:49.066115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:37:49.067414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:49.067414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:49.067414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:49.067414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:49.067414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:49.067414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:49.067414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:49.067414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:49.067414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:49.067414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:49.067414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:49.067414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:49.067414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:49.067414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:49.067414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:49.067414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:49.067414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:37:49.068474 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:37:49.068474 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:49.068474 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:49.068474 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 259/508] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:49.069742 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e9727-6d5e-4bac-9762-500fd24139b3 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:49.097895 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4394f-5330-4fac-87bb-42d168575302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:49.098755 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4394f-5330-4fac-87bb-42d168575302 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:49.098755 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4394f-5330-4fac-87bb-42d168575302 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:49.099249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4394f-5330-4fac-87bb-42d168575302 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:49.175583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf4394f-5330-4fac-87bb-42d168575302 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:49.178515 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf4394f-5330-4fac-87bb-42d168575302 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:49.182743 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e9727-6d5e-4bac-9762-500fd24139b3 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:49.210013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:49.221265 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8912c5-a5e8-41fa-a468-fb33a3b4a28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:49.221877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8912c5-a5e8-41fa-a468-fb33a3b4a28c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:49.222152 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8912c5-a5e8-41fa-a468-fb33a3b4a28c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:49.222420 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8912c5-a5e8-41fa-a468-fb33a3b4a28c None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:49.223193 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8912c5-a5e8-41fa-a468-fb33a3b4a28c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:49.223494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8912c5-a5e8-41fa-a468-fb33a3b4a28c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:49.224948 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8912c5-a5e8-41fa-a468-fb33a3b4a28c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:49.271204 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 260/509] 10.222.0.71 () {56 vars in 1137 bytes} [Fri Aug 26 08:37:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:49.272332 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8912c5-a5e8-41fa-a468-fb33a3b4a28c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:49.291529 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:49.296366 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 250/510] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:37:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:37:49.297216 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4394f-5330-4fac-87bb-42d168575302 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:49.388451 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8912c5-a5e8-41fa-a468-fb33a3b4a28c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:49.405889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4394f-5330-4fac-87bb-42d168575302 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:49.424509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3d119-324b-47c7-9df5-8f8361bef699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:49.425187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3d119-324b-47c7-9df5-8f8361bef699 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:49.425558 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3d119-324b-47c7-9df5-8f8361bef699 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:49.425764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3d119-324b-47c7-9df5-8f8361bef699 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:49.436442 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae3d119-324b-47c7-9df5-8f8361bef699 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:49.437913 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae3d119-324b-47c7-9df5-8f8361bef699 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:49.439607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888d8b1-74ea-416f-96f6-47b4994eb5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:49.440233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888d8b1-74ea-416f-96f6-47b4994eb5f0 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:49.440531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888d8b1-74ea-416f-96f6-47b4994eb5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:49.440790 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888d8b1-74ea-416f-96f6-47b4994eb5f0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:49.449249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6888d8b1-74ea-416f-96f6-47b4994eb5f0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:49.450729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6888d8b1-74ea-416f-96f6-47b4994eb5f0 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:49.456377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-9ae3d119-324b-47c7-9df5-8f8361bef699 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:49.456377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:49.456377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:49.456377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:49.456377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:49.456377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:49.456377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:49.456377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:49.456377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:49.456377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:49.456377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:49.456377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:49.456377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:37:49.456377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:37:49.456377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:37:49.456377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:37:49.458341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:49.458341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:49.458341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:49.458341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:49.458341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:49.458341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:49.458341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:49.458341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:49.458341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:49.458341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:49.458341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:49.458341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:49.458341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:49.458341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:49.458341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:49.458341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:49.458341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:37:49.459295 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:37:49.459295 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:49.459295 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:49.459295 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 261/511] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:49.459295 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3d119-324b-47c7-9df5-8f8361bef699 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:49.465557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-6888d8b1-74ea-416f-96f6-47b4994eb5f0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 08:37:49.465557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:49.465557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:49.465557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:49.465557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:49.465557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:49.465557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:49.465557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:49.465557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:49.465557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:49.465557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:49.465557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:49.465557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:37:49.465557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:37:49.465557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:37:49.465557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:37:49.466645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:49.466645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:49.466645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:37:49.466645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:37:49.466645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:37:49.466645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:37:49.466645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:49.466645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:49.466645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:49.466645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:49.466645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:49.466645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:49.466645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:49.466645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:49.466645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:49.466645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:49.466645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:49.466645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:49.467588 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:49.467588 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:49.467588 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:37:49.467588 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:37:49.467588 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:37:49.467588 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:37:49.467588 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:37:49.467588 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:37:49.467588 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 08:37:49.467588 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:49.467588 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 251/512] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:37:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:49.468271 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888d8b1-74ea-416f-96f6-47b4994eb5f0 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:49.500064 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:49.532598 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:49.572851 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3d119-324b-47c7-9df5-8f8361bef699 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:49.581456 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888d8b1-74ea-416f-96f6-47b4994eb5f0 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:49.615226 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e9948-a184-4cb8-a031-43166eac164a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:49.616033 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e9948-a184-4cb8-a031-43166eac164a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:49.616450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e9948-a184-4cb8-a031-43166eac164a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:49.617050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e9948-a184-4cb8-a031-43166eac164a None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:49.617415 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e0debe-7844-4920-91fb-d8374f502681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:49.617792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e9948-a184-4cb8-a031-43166eac164a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:49.618023 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0debe-7844-4920-91fb-d8374f502681 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:49.618188 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e9948-a184-4cb8-a031-43166eac164a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:49.618335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0debe-7844-4920-91fb-d8374f502681 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:49.618553 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0debe-7844-4920-91fb-d8374f502681 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:49.619635 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e9948-a184-4cb8-a031-43166eac164a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:49.633183 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 262/513] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:49.634420 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e9948-a184-4cb8-a031-43166eac164a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:49.698696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e0debe-7844-4920-91fb-d8374f502681 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:49.700127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e0debe-7844-4920-91fb-d8374f502681 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:49.734827 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:49.741628 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e9948-a184-4cb8-a031-43166eac164a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:49.762488 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:49.777712 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f29c3-7146-4678-83df-95f178430f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:49.778342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f29c3-7146-4678-83df-95f178430f22 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:49.778680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f29c3-7146-4678-83df-95f178430f22 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:49.778912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f29c3-7146-4678-83df-95f178430f22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:49.789964 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56f29c3-7146-4678-83df-95f178430f22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:49.791482 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56f29c3-7146-4678-83df-95f178430f22 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:49.806910 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 252/514] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:37:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:37:49.807086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-f56f29c3-7146-4678-83df-95f178430f22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:49.807086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:49.807086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:49.807086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:49.807086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:49.807086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:49.807086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:49.807086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:49.807086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:49.807086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:49.807086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:49.807086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:49.807086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:37:49.807086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:37:49.807086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:37:49.807086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:37:49.812113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:49.812113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:49.812113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:49.812113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:49.812113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:49.812113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:49.812113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:49.812113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:49.812113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:49.812113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:49.812113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:49.812113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:49.812113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:49.812113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:49.812113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:49.812113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:49.812113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:37:49.813205 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:37:49.813205 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:49.813205 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:49.813415 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e0debe-7844-4920-91fb-d8374f502681 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:49.813501 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 263/515] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:49.813501 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f29c3-7146-4678-83df-95f178430f22 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:49.917595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e0debe-7844-4920-91fb-d8374f502681 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:49.917852 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f29c3-7146-4678-83df-95f178430f22 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:49.954610 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f9da5-a362-4551-a74e-3f6dd69eb3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:49.955095 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f9da5-a362-4551-a74e-3f6dd69eb3c8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:49.955377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f9da5-a362-4551-a74e-3f6dd69eb3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:49.955650 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f9da5-a362-4551-a74e-3f6dd69eb3c8 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:49.956388 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f9da5-a362-4551-a74e-3f6dd69eb3c8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:49.956692 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f9da5-a362-4551-a74e-3f6dd69eb3c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:49.958219 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f9da5-a362-4551-a74e-3f6dd69eb3c8 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:49.964509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a6e3d-1208-44ee-89f6-58cb56bb5376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:49.965140 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a6e3d-1208-44ee-89f6-58cb56bb5376 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:49.965456 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a6e3d-1208-44ee-89f6-58cb56bb5376 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:49.965730 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a6e3d-1208-44ee-89f6-58cb56bb5376 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:49.966399 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a6e3d-1208-44ee-89f6-58cb56bb5376 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:49.966699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a6e3d-1208-44ee-89f6-58cb56bb5376 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:49.968130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a6e3d-1208-44ee-89f6-58cb56bb5376 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:49.976598 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 264/516] 10.222.0.71 () {56 vars in 1137 bytes} [Fri Aug 26 08:37:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:49.977469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f9da5-a362-4551-a74e-3f6dd69eb3c8 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:49.980204 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 253/517] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:49.981033 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a6e3d-1208-44ee-89f6-58cb56bb5376 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:49.986181 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:50.001377 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:50.085905 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f9da5-a362-4551-a74e-3f6dd69eb3c8 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:50.092378 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a6e3d-1208-44ee-89f6-58cb56bb5376 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:50.121423 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c869c-4fae-4c68-835d-118c666a560a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:50.122055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c869c-4fae-4c68-835d-118c666a560a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:50.122339 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c869c-4fae-4c68-835d-118c666a560a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:50.122601 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c869c-4fae-4c68-835d-118c666a560a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:50.127523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b50f0-4260-4c39-8b9c-f5745e96dce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:50.128196 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b50f0-4260-4c39-8b9c-f5745e96dce2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:50.128455 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b50f0-4260-4c39-8b9c-f5745e96dce2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:50.128761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b50f0-4260-4c39-8b9c-f5745e96dce2 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:50.129584 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b50f0-4260-4c39-8b9c-f5745e96dce2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:50.129839 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b50f0-4260-4c39-8b9c-f5745e96dce2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:50.130607 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c869c-4fae-4c68-835d-118c666a560a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:50.131285 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b50f0-4260-4c39-8b9c-f5745e96dce2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:50.132078 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c869c-4fae-4c68-835d-118c666a560a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:50.144280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-221c869c-4fae-4c68-835d-118c666a560a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 08:37:50.144280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:50.144280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:50.144280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:50.144280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:50.144280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:50.144280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:50.144280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:50.144280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:50.144280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:50.144280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:50.144280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:50.144280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:37:50.144280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:37:50.144280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:37:50.144280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:37:50.145327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:50.145327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:50.145327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:37:50.145327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:37:50.145327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:37:50.145327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:37:50.145327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:50.145327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:50.145327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:50.145327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:50.145327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:50.145327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:50.145327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:50.145327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:50.145327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:50.145327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:50.145327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:50.145327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:50.151302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:50.151302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:50.151302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:37:50.151302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:37:50.151302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:37:50.151302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:37:50.151302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:37:50.151302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:37:50.151302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 08:37:50.151302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:50.151810 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 254/518] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:50.151887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 265/519] 10.222.0.71 () {56 vars in 1102 bytes} [Fri Aug 26 08:37:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:50.151957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b50f0-4260-4c39-8b9c-f5745e96dce2 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:50.152028 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c869c-4fae-4c68-835d-118c666a560a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:50.218805 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:50.234518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:50.260105 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b50f0-4260-4c39-8b9c-f5745e96dce2 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:50.261736 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c869c-4fae-4c68-835d-118c666a560a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:50.295613 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca267a0-8f83-4618-848d-a6133642cdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:50.296243 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca267a0-8f83-4618-848d-a6133642cdfe None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:50.296543 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca267a0-8f83-4618-848d-a6133642cdfe None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:50.296804 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca267a0-8f83-4618-848d-a6133642cdfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:50.298509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf8216-3ca0-4d84-af3f-9c19fd399182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:50.299162 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf8216-3ca0-4d84-af3f-9c19fd399182 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:50.299421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf8216-3ca0-4d84-af3f-9c19fd399182 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:50.299695 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf8216-3ca0-4d84-af3f-9c19fd399182 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:50.308206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca267a0-8f83-4618-848d-a6133642cdfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:50.309749 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca267a0-8f83-4618-848d-a6133642cdfe None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:50.325233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-4ca267a0-8f83-4618-848d-a6133642cdfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:50.325233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:50.325233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:50.325233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:50.325233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:50.325233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:50.325233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:50.325233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:50.325233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:50.325233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:50.325233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:50.325233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:50.325233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:37:50.325233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:37:50.325233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:37:50.325233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:37:50.326211 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:50.326211 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:50.326211 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:50.326211 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:50.326211 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:50.326211 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:50.326211 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:50.326211 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:50.326211 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:50.326211 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:50.326211 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:50.326211 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:50.326211 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:50.326211 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:50.326211 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:50.326211 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:50.326211 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:37:50.327102 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:37:50.327102 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:50.327102 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:50.327102 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 255/520] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:50.327313 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca267a0-8f83-4618-848d-a6133642cdfe None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:50.373831 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf8216-3ca0-4d84-af3f-9c19fd399182 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:50.375333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf8216-3ca0-4d84-af3f-9c19fd399182 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:50.436031 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca267a0-8f83-4618-848d-a6133642cdfe None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:50.437929 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:50.468072 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:50.475394 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbabc24-4601-4c89-8b16-02146fab6264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:50.475943 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbabc24-4601-4c89-8b16-02146fab6264 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:50.476225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbabc24-4601-4c89-8b16-02146fab6264 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:50.476510 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbabc24-4601-4c89-8b16-02146fab6264 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:50.477149 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbabc24-4601-4c89-8b16-02146fab6264 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:50.478619 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbabc24-4601-4c89-8b16-02146fab6264 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:50.481912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 266/521] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:37:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:37:50.482759 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf8216-3ca0-4d84-af3f-9c19fd399182 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:50.505227 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 256/522] 10.222.0.71 () {60 vars in 1132 bytes} [Fri Aug 26 08:37:50 2022] POST /identity/v3/users => generated 247 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:37:50.506122 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbabc24-4601-4c89-8b16-02146fab6264 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:50.615247 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf8216-3ca0-4d84-af3f-9c19fd399182 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:50.643744 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbabc24-4601-4c89-8b16-02146fab6264 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:50.661177 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71121dd4-6512-4741-bfb1-7b8c391c4fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:50.661681 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71121dd4-6512-4741-bfb1-7b8c391c4fc5 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:50.661962 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71121dd4-6512-4741-bfb1-7b8c391c4fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:50.662234 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71121dd4-6512-4741-bfb1-7b8c391c4fc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:50.670315 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71121dd4-6512-4741-bfb1-7b8c391c4fc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:50.671826 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71121dd4-6512-4741-bfb1-7b8c391c4fc5 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:50.685042 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-71121dd4-6512-4741-bfb1-7b8c391c4fc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:37:50.685042 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:50.685042 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:50.685042 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:50.685042 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:50.685042 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:50.685042 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:50.685042 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:50.685042 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:50.685042 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:50.685042 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:50.685042 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:50.685042 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:37:50.685042 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:37:50.685042 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:37:50.685990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:37:50.685990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:50.685990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:50.685990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:50.685990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:50.685990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:50.685990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:50.685990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:50.685990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:50.685990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:50.685990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:50.685990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:50.685990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:50.685990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:50.685990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:50.685990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:50.685990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:50.686861 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:37:50.686861 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:37:50.686861 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:37:50.686861 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:37:50.686861 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:37:50.686861 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:37:50.686861 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:37:50.686861 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:50.687368 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb5e4c-4fd5-4ff8-8968-74301439a7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:50.687368 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb5e4c-4fd5-4ff8-8968-74301439a7e8 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:50.687547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 267/523] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:37:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:50.687628 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb5e4c-4fd5-4ff8-8968-74301439a7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:50.687628 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb5e4c-4fd5-4ff8-8968-74301439a7e8 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:50.687762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71121dd4-6512-4741-bfb1-7b8c391c4fc5 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:50.687892 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eb5e4c-4fd5-4ff8-8968-74301439a7e8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:50.687992 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eb5e4c-4fd5-4ff8-8968-74301439a7e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:50.689417 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eb5e4c-4fd5-4ff8-8968-74301439a7e8 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:50.713687 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 257/524] 10.222.0.71 () {56 vars in 1139 bytes} [Fri Aug 26 08:37:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:50.714556 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb5e4c-4fd5-4ff8-8968-74301439a7e8 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:50.719146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:50.764839 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:50.806196 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71121dd4-6512-4741-bfb1-7b8c391c4fc5 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:50.825653 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb5e4c-4fd5-4ff8-8968-74301439a7e8 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:50.849494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99907818-f975-4dde-b7e1-db35d13c3a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:50.850110 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99907818-f975-4dde-b7e1-db35d13c3a6a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:50.850388 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99907818-f975-4dde-b7e1-db35d13c3a6a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:50.850657 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99907818-f975-4dde-b7e1-db35d13c3a6a None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:50.851345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99907818-f975-4dde-b7e1-db35d13c3a6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:50.851644 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99907818-f975-4dde-b7e1-db35d13c3a6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:50.853113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99907818-f975-4dde-b7e1-db35d13c3a6a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:50.871775 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 268/525] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:50.871775 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99907818-f975-4dde-b7e1-db35d13c3a6a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:50.874509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a8386-0b83-4bf9-84da-13891830e92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:50.875412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a8386-0b83-4bf9-84da-13891830e92e None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:50.876609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a8386-0b83-4bf9-84da-13891830e92e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:50.876899 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a8386-0b83-4bf9-84da-13891830e92e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:50.890865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a8386-0b83-4bf9-84da-13891830e92e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:50.892369 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a8386-0b83-4bf9-84da-13891830e92e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:50.910069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-240a8386-0b83-4bf9-84da-13891830e92e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:50.910069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:50.910069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:50.910069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:50.910069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:50.910069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:50.910069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:50.910069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:50.910069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:50.910069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:50.910069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:50.910069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:50.910069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:37:50.910069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:37:50.910069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:37:50.910069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:37:50.911127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:50.911127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:50.911127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:50.911127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:50.911127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:50.911127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:50.911127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:50.911127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:50.911127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:50.911127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:50.911127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:50.911127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:50.911127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:50.911127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:50.911127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:50.911127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:50.911127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:37:50.912175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:37:50.912175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:50.912175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:50.918034 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 258/526] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:50.918034 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a8386-0b83-4bf9-84da-13891830e92e None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:50.967540 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:50.986087 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99907818-f975-4dde-b7e1-db35d13c3a6a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:51.039789 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2cc384-6100-4076-a3f2-29d6c2991660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:51.040367 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2cc384-6100-4076-a3f2-29d6c2991660 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:51.040802 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2cc384-6100-4076-a3f2-29d6c2991660 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:51.041285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2cc384-6100-4076-a3f2-29d6c2991660 None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:51.042579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2cc384-6100-4076-a3f2-29d6c2991660 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:51.043051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:51.043213 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2cc384-6100-4076-a3f2-29d6c2991660 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:51.045517 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2cc384-6100-4076-a3f2-29d6c2991660 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:51.050162 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a8386-0b83-4bf9-84da-13891830e92e None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:51.077379 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 269/527] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:37:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:51.078243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2cc384-6100-4076-a3f2-29d6c2991660 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:51.099660 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fe6fc-32b0-4f6b-ae36-8c924b462bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:51.101110 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fe6fc-32b0-4f6b-ae36-8c924b462bb5 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:51.101855 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fe6fc-32b0-4f6b-ae36-8c924b462bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:51.102532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fe6fc-32b0-4f6b-ae36-8c924b462bb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:51.203195 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fe6fc-32b0-4f6b-ae36-8c924b462bb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:51.205315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fe6fc-32b0-4f6b-ae36-8c924b462bb5 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:51.221163 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2cc384-6100-4076-a3f2-29d6c2991660 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:51.273629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd1b037-05f2-4800-884c-a278a485fa48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:51.274696 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1b037-05f2-4800-884c-a278a485fa48 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:51.275030 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1b037-05f2-4800-884c-a278a485fa48 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:51.275306 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1b037-05f2-4800-884c-a278a485fa48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:51.283778 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd1b037-05f2-4800-884c-a278a485fa48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:51.285525 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd1b037-05f2-4800-884c-a278a485fa48 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:51.298710 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-efd1b037-05f2-4800-884c-a278a485fa48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:37:51.298710 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:51.298710 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:51.298710 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:51.298710 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:51.298710 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:51.298710 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:51.298710 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:51.298710 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:51.298710 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:51.298710 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:51.298710 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:51.298710 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:37:51.298710 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:37:51.298710 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:37:51.299680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:37:51.299680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:51.299680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:51.299680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:51.299680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:51.299680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:51.299680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:51.299680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:51.299680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:51.299680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:51.299680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:51.299680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:51.299680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:51.299680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:51.299680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:51.299680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:51.299680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:51.300570 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:37:51.300570 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:37:51.300570 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:37:51.300570 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:37:51.300570 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:37:51.300570 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:37:51.300570 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:37:51.300570 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:51.300570 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 270/528] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:37:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:51.302216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd1b037-05f2-4800-884c-a278a485fa48 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:51.309784 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:51.326259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:51.349039 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 259/529] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:37:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:37:51.354326 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fe6fc-32b0-4f6b-ae36-8c924b462bb5 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:51.435516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd1b037-05f2-4800-884c-a278a485fa48 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:51.496324 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ec868-eb48-460c-87e3-0a7849fdf6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:51.496977 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ec868-eb48-460c-87e3-0a7849fdf6c4 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:51.497252 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ec868-eb48-460c-87e3-0a7849fdf6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:51.497487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ec868-eb48-460c-87e3-0a7849fdf6c4 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:51.498151 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ec868-eb48-460c-87e3-0a7849fdf6c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:51.498569 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ec868-eb48-460c-87e3-0a7849fdf6c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:51.500041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ec868-eb48-460c-87e3-0a7849fdf6c4 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:51.503482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fe6fc-32b0-4f6b-ae36-8c924b462bb5 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:51.518932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 271/530] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:37:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:51.518932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ec868-eb48-460c-87e3-0a7849fdf6c4 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:51.570040 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d27146-2507-48a8-aa82-ad07572810ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:51.571055 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d27146-2507-48a8-aa82-ad07572810ca None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:51.571521 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d27146-2507-48a8-aa82-ad07572810ca None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:51.571960 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d27146-2507-48a8-aa82-ad07572810ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:51.573180 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d27146-2507-48a8-aa82-ad07572810ca None admin] RBAC: Adding query filter params (role.id=33ac296b51c143e7bb44211175752edb, scope.project.id=543d44293d894a7fbb83401f7a26dd9b, user.id=f16d1768715a41648af3fe6afadd7477) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:51.575033 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d27146-2507-48a8-aa82-ad07572810ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:51.575353 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d27146-2507-48a8-aa82-ad07572810ca None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:51.593037 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 260/531] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:37:51 2022] GET /identity/v3/role_assignments?user.id=f16d1768715a41648af3fe6afadd7477&scope.project.id=543d44293d894a7fbb83401f7a26dd9b&role.id=33ac296b51c143e7bb44211175752edb => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:51.593941 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d27146-2507-48a8-aa82-ad07572810ca None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:51.644301 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:51.690964 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:51.714448 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ec868-eb48-460c-87e3-0a7849fdf6c4 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:51.764341 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d27146-2507-48a8-aa82-ad07572810ca None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:51.768663 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88049161-78e0-4e50-90be-25fdb5f1ecce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:51.769658 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88049161-78e0-4e50-90be-25fdb5f1ecce None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:51.770149 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88049161-78e0-4e50-90be-25fdb5f1ecce None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:51.770595 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88049161-78e0-4e50-90be-25fdb5f1ecce None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:51.771704 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88049161-78e0-4e50-90be-25fdb5f1ecce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:51.772020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88049161-78e0-4e50-90be-25fdb5f1ecce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:51.774957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88049161-78e0-4e50-90be-25fdb5f1ecce None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:51.790077 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 272/532] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:37:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:51.791052 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88049161-78e0-4e50-90be-25fdb5f1ecce None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:51.805251 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35ffa4-2abf-4831-919f-7b1c31389096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:51.806235 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35ffa4-2abf-4831-919f-7b1c31389096 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:51.806730 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35ffa4-2abf-4831-919f-7b1c31389096 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:51.807193 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35ffa4-2abf-4831-919f-7b1c31389096 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:51.820595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35ffa4-2abf-4831-919f-7b1c31389096 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:51.822566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35ffa4-2abf-4831-919f-7b1c31389096 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:51.839099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-5d35ffa4-2abf-4831-919f-7b1c31389096 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:51.839099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:51.839099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:51.839099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:51.839099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:51.839099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:51.839099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:51.839099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:51.839099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:51.839099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:51.839099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:51.839099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:51.839099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:37:51.839099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:37:51.839099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:37:51.839099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:37:51.840070 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:51.840070 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:51.840070 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:51.840070 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:51.840070 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:51.840070 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:51.840070 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:51.840070 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:51.840070 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:51.840070 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:51.840070 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:51.840070 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:51.840070 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:51.840070 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:51.840070 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:51.840070 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:51.840070 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:37:51.841045 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:37:51.841045 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:51.841045 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:51.848120 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 261/533] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:51.849464 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35ffa4-2abf-4831-919f-7b1c31389096 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:51.915028 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88049161-78e0-4e50-90be-25fdb5f1ecce None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:51.941871 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:51.954395 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55847b68-7533-4c05-9042-311bf0d4078b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:51.955109 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55847b68-7533-4c05-9042-311bf0d4078b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:51.955390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55847b68-7533-4c05-9042-311bf0d4078b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:51.955672 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55847b68-7533-4c05-9042-311bf0d4078b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:51.964529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55847b68-7533-4c05-9042-311bf0d4078b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:51.966093 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55847b68-7533-4c05-9042-311bf0d4078b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:51.969131 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35ffa4-2abf-4831-919f-7b1c31389096 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:51.980091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-55847b68-7533-4c05-9042-311bf0d4078b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:37:51.980091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:51.980091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:51.980091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:51.980091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:51.980091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:51.980091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:51.980091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:51.980091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:51.980091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:51.980091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:51.980091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:51.980091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:37:51.980091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:37:51.980091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:37:51.981837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:37:51.981837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:51.981837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:51.981837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:51.981837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:51.981837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:51.981837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:51.981837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:51.981837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:51.981837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:51.981837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:51.981837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:51.981837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:51.981837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:51.981837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:51.981837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:51.981837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:51.983551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:37:51.983551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:37:51.983551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:37:51.983551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:37:51.983551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:37:51.983551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:37:51.983551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:37:51.983551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:51.983551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 273/534] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:37:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:51.983551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55847b68-7533-4c05-9042-311bf0d4078b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:51.999329 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:52.037933 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf9fa5-ae84-4566-ba2a-072b4c6a1393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:52.039531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf9fa5-ae84-4566-ba2a-072b4c6a1393 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:52.040232 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf9fa5-ae84-4566-ba2a-072b4c6a1393 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:52.040964 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf9fa5-ae84-4566-ba2a-072b4c6a1393 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:52.042845 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf9fa5-ae84-4566-ba2a-072b4c6a1393 None admin] RBAC: Adding query filter params (role.id=33ac296b51c143e7bb44211175752edb, scope.project.id=543d44293d894a7fbb83401f7a26dd9b, user.id=0d178c717a3e49319545e70cb58cbbed) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:52.043768 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf9fa5-ae84-4566-ba2a-072b4c6a1393 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:52.047102 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf9fa5-ae84-4566-ba2a-072b4c6a1393 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:52.058792 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 262/535] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:37:51 2022] GET /identity/v3/role_assignments?user.id=0d178c717a3e49319545e70cb58cbbed&scope.project.id=543d44293d894a7fbb83401f7a26dd9b&role.id=33ac296b51c143e7bb44211175752edb => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:52.061071 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf9fa5-ae84-4566-ba2a-072b4c6a1393 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:52.094257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55847b68-7533-4c05-9042-311bf0d4078b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:52.175791 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538035f7-efba-4c95-b9f3-b8d4185e184a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:52.176538 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538035f7-efba-4c95-b9f3-b8d4185e184a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:52.176840 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538035f7-efba-4c95-b9f3-b8d4185e184a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:52.177110 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538035f7-efba-4c95-b9f3-b8d4185e184a None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:52.177783 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538035f7-efba-4c95-b9f3-b8d4185e184a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:52.178092 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538035f7-efba-4c95-b9f3-b8d4185e184a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:52.179540 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538035f7-efba-4c95-b9f3-b8d4185e184a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:52.194511 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 274/536] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:37:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:52.208144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538035f7-efba-4c95-b9f3-b8d4185e184a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:52.266041 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf9fa5-ae84-4566-ba2a-072b4c6a1393 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:52.298043 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:52.317786 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6526dd3-1550-4b0c-a866-93792d1e4d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:52.318446 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6526dd3-1550-4b0c-a866-93792d1e4d9e None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:52.318722 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6526dd3-1550-4b0c-a866-93792d1e4d9e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:52.319021 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6526dd3-1550-4b0c-a866-93792d1e4d9e None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:52.319731 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6526dd3-1550-4b0c-a866-93792d1e4d9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:52.320040 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6526dd3-1550-4b0c-a866-93792d1e4d9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:52.321514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6526dd3-1550-4b0c-a866-93792d1e4d9e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:52.342519 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 263/537] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:37:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:52.348053 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6526dd3-1550-4b0c-a866-93792d1e4d9e None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:52.419460 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:52.421312 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538035f7-efba-4c95-b9f3-b8d4185e184a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:52.475728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c5a8a-d34c-460b-96b8-71e49f433e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:52.476428 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c5a8a-d34c-460b-96b8-71e49f433e11 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:52.476714 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c5a8a-d34c-460b-96b8-71e49f433e11 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:52.476994 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c5a8a-d34c-460b-96b8-71e49f433e11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:52.492642 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c5a8a-d34c-460b-96b8-71e49f433e11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:52.494257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c5a8a-d34c-460b-96b8-71e49f433e11 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:52.522318 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6526dd3-1550-4b0c-a866-93792d1e4d9e None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:52.533294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-6a4c5a8a-d34c-460b-96b8-71e49f433e11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:37:52.533294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:52.533294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:52.533294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:52.533294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:52.533294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:52.533294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:52.533294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:52.533294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:52.533294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:52.533294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:52.533294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:52.533294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:37:52.533294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:37:52.533294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:37:52.534565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:37:52.534565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:52.534565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:52.534565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:52.534565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:52.534565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:52.534565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:52.534565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:52.534565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:52.534565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:52.534565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:52.534565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:52.534565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:52.534565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:52.534565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:52.534565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:52.534565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:52.535464 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:37:52.535464 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:37:52.535464 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:37:52.535464 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:37:52.535464 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:37:52.535464 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:37:52.535464 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:37:52.535464 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:52.543435 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 275/538] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:37:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:52.549696 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c5a8a-d34c-460b-96b8-71e49f433e11 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:52.575386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb274ec-f5f9-417c-97cd-db012e868bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:52.576011 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb274ec-f5f9-417c-97cd-db012e868bbe None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:52.576285 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb274ec-f5f9-417c-97cd-db012e868bbe None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:52.576553 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb274ec-f5f9-417c-97cd-db012e868bbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:52.577330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb274ec-f5f9-417c-97cd-db012e868bbe None admin] RBAC: Adding query filter params (role.id=33ac296b51c143e7bb44211175752edb, scope.project.id=543d44293d894a7fbb83401f7a26dd9b, user.id=997b4ffc01fc467ebbd1092787f217f6) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:52.577636 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb274ec-f5f9-417c-97cd-db012e868bbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:52.579165 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb274ec-f5f9-417c-97cd-db012e868bbe None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:52.596507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 264/539] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:37:52 2022] GET /identity/v3/role_assignments?user.id=997b4ffc01fc467ebbd1092787f217f6&scope.project.id=543d44293d894a7fbb83401f7a26dd9b&role.id=33ac296b51c143e7bb44211175752edb => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:37:52.615994 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b54b6-4aaf-4622-9390-503435bebc3e None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:52.616171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b54b6-4aaf-4622-9390-503435bebc3e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:52.616429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b54b6-4aaf-4622-9390-503435bebc3e None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:52.618503 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 265/540] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:37:52.636485 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95dd1a-f60a-4ad1-baf6-95238e538aad None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:52.636808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95dd1a-f60a-4ad1-baf6-95238e538aad None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:52.637024 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95dd1a-f60a-4ad1-baf6-95238e538aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:52.676416 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ba95dd1a-f60a-4ad1-baf6-95238e538aad None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:52.678013 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ba95dd1a-f60a-4ad1-baf6-95238e538aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:52.713857 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c5a8a-d34c-460b-96b8-71e49f433e11 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:52.724349 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:52.774491 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03a5ea-328d-4de8-a27f-c7bf8d88608e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:52.775156 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03a5ea-328d-4de8-a27f-c7bf8d88608e None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:52.775438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03a5ea-328d-4de8-a27f-c7bf8d88608e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:52.775711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03a5ea-328d-4de8-a27f-c7bf8d88608e None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:52.776440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd03a5ea-328d-4de8-a27f-c7bf8d88608e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:52.776749 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd03a5ea-328d-4de8-a27f-c7bf8d88608e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:52.778249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd03a5ea-328d-4de8-a27f-c7bf8d88608e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:52.803699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 276/541] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:37:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:37:52.812054 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03a5ea-328d-4de8-a27f-c7bf8d88608e None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:52.858515 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 266/542] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:52.987768 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03a5ea-328d-4de8-a27f-c7bf8d88608e None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:53.043090 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0a484-22e3-43b7-9600-4d422dea2896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:53.043807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0a484-22e3-43b7-9600-4d422dea2896 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:53.044130 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0a484-22e3-43b7-9600-4d422dea2896 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:53.044405 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0a484-22e3-43b7-9600-4d422dea2896 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:53.045148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a0a484-22e3-43b7-9600-4d422dea2896 None admin] RBAC: Adding query filter params (role.id=33ac296b51c143e7bb44211175752edb, scope.project.id=543d44293d894a7fbb83401f7a26dd9b, user.id=4aa8938a504c4c4f8815478e520482d8) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:53.045567 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a0a484-22e3-43b7-9600-4d422dea2896 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:53.047100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a0a484-22e3-43b7-9600-4d422dea2896 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:53.057190 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 277/543] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:37:52 2022] GET /identity/v3/role_assignments?user.id=4aa8938a504c4c4f8815478e520482d8&scope.project.id=543d44293d894a7fbb83401f7a26dd9b&role.id=33ac296b51c143e7bb44211175752edb => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:53.235905 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95dd1a-f60a-4ad1-baf6-95238e538aad None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:53.287168 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:53.408575 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95dd1a-f60a-4ad1-baf6-95238e538aad None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:53.458111 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8a9b0-103c-4a2b-b9fe-5829f0314b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:53.458863 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8a9b0-103c-4a2b-b9fe-5829f0314b8d None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:53.459365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8a9b0-103c-4a2b-b9fe-5829f0314b8d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:53.459845 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8a9b0-103c-4a2b-b9fe-5829f0314b8d None admin] PATH_INFO: `/v3/roles/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:53.480363 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e8a9b0-103c-4a2b-b9fe-5829f0314b8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:53.485451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-71e8a9b0-103c-4a2b-b9fe-5829f0314b8d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:37:53.485451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:53.485451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:53.485451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:53.485451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:53.485451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:53.485451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:53.485451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:53.485451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:53.485451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:53.485451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:53.485451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:53.485451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:53.485451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:53.485451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:53.485451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:53.486335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:53.486335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:53.486335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:53.486335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:53.486335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:53.486335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:53.486335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:53.486335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:53.486335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:53.486335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:53.486335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:53.486335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:53.486335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:53.486335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:53.486335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:53.486335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:53.487186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:53.487186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:53.487186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:53.487186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:53.487186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:53.487186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:53.487186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:53.487186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:53.487186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:53.487186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:53.487186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:37:53.487186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:53.489503 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 267/544] 10.222.0.71 () {56 vars in 1102 bytes} [Fri Aug 26 08:37:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:53.496691 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0a484-22e3-43b7-9600-4d422dea2896 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:53.547893 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:53.665008 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0a484-22e3-43b7-9600-4d422dea2896 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:53.712339 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b286780-4468-4aa6-ab90-1d10846dcfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:53.712991 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b286780-4468-4aa6-ab90-1d10846dcfb8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:53.713268 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b286780-4468-4aa6-ab90-1d10846dcfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:53.713537 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b286780-4468-4aa6-ab90-1d10846dcfb8 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:53.714210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b286780-4468-4aa6-ab90-1d10846dcfb8 None admin] RBAC: Adding query filter params (name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:53.714607 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b286780-4468-4aa6-ab90-1d10846dcfb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:53.716235 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b286780-4468-4aa6-ab90-1d10846dcfb8 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:53.731309 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 278/545] 10.222.0.71 () {56 vars in 1103 bytes} [Fri Aug 26 08:37:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:37:53.737987 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8a9b0-103c-4a2b-b9fe-5829f0314b8d None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:53.921385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:53.961364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8a9b0-103c-4a2b-b9fe-5829f0314b8d None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:54.091440 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c663d87-9388-46f1-a280-4320f20beb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:54.092343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c663d87-9388-46f1-a280-4320f20beb30 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:54.092625 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c663d87-9388-46f1-a280-4320f20beb30 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:54.092895 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c663d87-9388-46f1-a280-4320f20beb30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:54.101801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd42914-85cf-4d3d-81f1-90e11013e67d None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:54.102247 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd42914-85cf-4d3d-81f1-90e11013e67d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:54.102676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd42914-85cf-4d3d-81f1-90e11013e67d None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:54.105552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 279/546] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:54.119342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1cbb3c-98fa-4e9f-9026-e6d7d7e33ea6 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:54.119789 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1cbb3c-98fa-4e9f-9026-e6d7d7e33ea6 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:54.120230 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1cbb3c-98fa-4e9f-9026-e6d7d7e33ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:54.123997 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c663d87-9388-46f1-a280-4320f20beb30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:54.125595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c663d87-9388-46f1-a280-4320f20beb30 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:54.143889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-6c663d87-9388-46f1-a280-4320f20beb30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:54.143889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:54.143889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:54.143889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:54.143889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:54.143889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:54.143889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:54.143889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:54.143889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:54.143889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:54.143889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:54.143889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:54.143889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:37:54.143889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:37:54.143889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:37:54.143889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:37:54.144962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:54.144962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:54.144962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:54.144962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:54.144962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:54.144962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:54.144962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:54.144962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:54.144962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:54.144962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:54.144962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:54.144962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:54.144962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:54.144962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:54.144962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:54.144962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:54.144962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:37:54.146757 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:37:54.146757 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:54.146757 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:54.146757 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 268/547] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:37:54.152854 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c663d87-9388-46f1-a280-4320f20beb30 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:54.163681 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-be1cbb3c-98fa-4e9f-9026-e6d7d7e33ea6 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:54.165560 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-be1cbb3c-98fa-4e9f-9026-e6d7d7e33ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:54.287142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 280/548] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 26 08:37:54.289473 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb58df-4f42-4bab-9c02-351dee591ae9 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:54.289973 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb58df-4f42-4bab-9c02-351dee591ae9 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:54.290914 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb58df-4f42-4bab-9c02-351dee591ae9 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:54.293620 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 281/549] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:54.300202 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b86ed6-21f9-4692-ba95-b66dd40b6e8d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:54.300936 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b86ed6-21f9-4692-ba95-b66dd40b6e8d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:54.301359 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b86ed6-21f9-4692-ba95-b66dd40b6e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:54.320457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c663d87-9388-46f1-a280-4320f20beb30 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:54.326954 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e2b86ed6-21f9-4692-ba95-b66dd40b6e8d None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:54.328405 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e2b86ed6-21f9-4692-ba95-b66dd40b6e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:54.356584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:54.384174 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155dfdc-0aed-4396-ad49-a776fbd19742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:54.385070 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155dfdc-0aed-4396-ad49-a776fbd19742 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:54.385070 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155dfdc-0aed-4396-ad49-a776fbd19742 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:54.385420 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155dfdc-0aed-4396-ad49-a776fbd19742 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:54.386580 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6155dfdc-0aed-4396-ad49-a776fbd19742 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:54.387172 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6155dfdc-0aed-4396-ad49-a776fbd19742 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:54.389757 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6155dfdc-0aed-4396-ad49-a776fbd19742 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:54.411470 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 269/550] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:37:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:54.424174 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155dfdc-0aed-4396-ad49-a776fbd19742 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:54.427765 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:54.476070 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 282/551] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:54.647215 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155dfdc-0aed-4396-ad49-a776fbd19742 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:54.702476 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96a206-c485-48f3-9120-de9ac1d3a558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:54.703425 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96a206-c485-48f3-9120-de9ac1d3a558 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:54.703860 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96a206-c485-48f3-9120-de9ac1d3a558 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:54.704365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d96a206-c485-48f3-9120-de9ac1d3a558 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:54.712548 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b86ed6-21f9-4692-ba95-b66dd40b6e8d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:54.718140 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d96a206-c485-48f3-9120-de9ac1d3a558 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:54.719920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d96a206-c485-48f3-9120-de9ac1d3a558 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:54.752904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-2d96a206-c485-48f3-9120-de9ac1d3a558 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 08:37:54.752904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:54.752904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:54.752904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:54.752904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:54.752904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:54.752904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:54.752904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:54.752904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:54.752904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:54.752904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:54.752904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:54.752904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:37:54.752904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:37:54.752904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:37:54.752904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:37:54.754969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:54.754969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:54.754969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:37:54.754969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:37:54.754969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:37:54.754969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:37:54.754969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:54.754969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:54.754969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:54.754969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:54.754969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:54.754969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:54.754969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:54.754969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:54.754969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:54.754969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:54.754969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:54.754969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:54.756627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:54.756627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:54.756627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:37:54.756627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:37:54.756627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:37:54.756627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:37:54.756627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:37:54.756627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:37:54.756627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 08:37:54.756627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:54.767034 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 270/552] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:37:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:54.770976 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96a206-c485-48f3-9120-de9ac1d3a558 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:54.882214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:54.911324 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b86ed6-21f9-4692-ba95-b66dd40b6e8d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:54.928185 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d96a206-c485-48f3-9120-de9ac1d3a558 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:54.950118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73466a-29a8-49db-adb9-6f8ff91e6fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:54.951182 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73466a-29a8-49db-adb9-6f8ff91e6fa9 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:54.951656 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73466a-29a8-49db-adb9-6f8ff91e6fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:54.952114 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73466a-29a8-49db-adb9-6f8ff91e6fa9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:54.963498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb73466a-29a8-49db-adb9-6f8ff91e6fa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:54.966778 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad4f326-9d62-4814-977f-e1587802daba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:54.967517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4f326-9d62-4814-977f-e1587802daba None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:54.967866 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4f326-9d62-4814-977f-e1587802daba None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:54.968209 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4f326-9d62-4814-977f-e1587802daba None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:54.969036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad4f326-9d62-4814-977f-e1587802daba None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:54.969409 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad4f326-9d62-4814-977f-e1587802daba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:54.969886 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-fb73466a-29a8-49db-adb9-6f8ff91e6fa9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:37:54.969886 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:54.969886 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:54.969886 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:54.969886 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:54.969886 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:54.969886 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:54.969886 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:54.969886 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:54.969886 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:54.969886 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:54.969886 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:54.969886 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:54.969886 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:54.969886 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:54.969886 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:54.970792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:54.970792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:54.970792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:54.970792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:54.970792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:54.970792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:54.970792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:54.970792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:54.970792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:54.970792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:54.970792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:54.970792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:54.970792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:54.970792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:54.970792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:54.970792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:54.971641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:54.971641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:54.971641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:54.971641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:54.971641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:54.971641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:54.971641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:54.971641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:54.971641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:54.971641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:54.971641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:37:54.971641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:54.972384 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad4f326-9d62-4814-977f-e1587802daba None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:54.979050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 283/553] 10.222.0.71 () {56 vars in 1102 bytes} [Fri Aug 26 08:37:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:54.979050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73466a-29a8-49db-adb9-6f8ff91e6fa9 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:54.994362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:55.003655 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 271/554] 10.222.0.71 () {56 vars in 1135 bytes} [Fri Aug 26 08:37:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:37:55.004557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad4f326-9d62-4814-977f-e1587802daba None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:55.100034 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73466a-29a8-49db-adb9-6f8ff91e6fa9 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:55.132952 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad4f326-9d62-4814-977f-e1587802daba None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:55.137622 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41908651-7b08-492e-a468-c5eb7aa188b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:55.138410 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41908651-7b08-492e-a468-c5eb7aa188b4 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:55.138759 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41908651-7b08-492e-a468-c5eb7aa188b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:55.139162 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41908651-7b08-492e-a468-c5eb7aa188b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:55.151284 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41908651-7b08-492e-a468-c5eb7aa188b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:55.156360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-41908651-7b08-492e-a468-c5eb7aa188b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:37:55.156360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:55.156360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:55.156360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:55.156360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:55.156360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:55.156360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:55.156360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:55.156360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:55.156360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:55.156360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:55.156360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:55.156360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:55.156360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:55.156360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:55.156360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:55.157273 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:55.157273 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:55.157273 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:55.157273 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:55.157273 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:55.157273 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:55.157273 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:55.157273 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:55.157273 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:55.157273 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:55.157273 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:55.157273 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:55.157273 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:55.157273 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:55.157273 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:55.157273 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:55.158082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:55.158082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:55.158082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:55.158082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:55.158082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:55.158082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:55.158082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:55.158082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:55.158082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:55.158082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:55.158082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:37:55.158082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:55.162777 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 284/555] 10.222.0.71 () {56 vars in 1102 bytes} [Fri Aug 26 08:37:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:55.163821 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41908651-7b08-492e-a468-c5eb7aa188b4 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:55.176237 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:55.180579 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f6e6c-24b5-4b66-8241-13209fd342d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:55.181279 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f6e6c-24b5-4b66-8241-13209fd342d6 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:55.181618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f6e6c-24b5-4b66-8241-13209fd342d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:55.181946 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f6e6c-24b5-4b66-8241-13209fd342d6 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:55.182978 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8f6e6c-24b5-4b66-8241-13209fd342d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:55.183343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8f6e6c-24b5-4b66-8241-13209fd342d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:55.185585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8f6e6c-24b5-4b66-8241-13209fd342d6 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:55.198974 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 272/556] 10.222.0.71 () {56 vars in 1103 bytes} [Fri Aug 26 08:37:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:55.200119 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f6e6c-24b5-4b66-8241-13209fd342d6 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:55.252047 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:55.296579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41908651-7b08-492e-a468-c5eb7aa188b4 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:55.329284 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8f6e6c-24b5-4b66-8241-13209fd342d6 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:55.335467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb643fd8-8991-49d2-9861-63139cf43a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:55.336413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb643fd8-8991-49d2-9861-63139cf43a19 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:55.336843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb643fd8-8991-49d2-9861-63139cf43a19 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:55.337365 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb643fd8-8991-49d2-9861-63139cf43a19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:55.376641 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51df3ad-fa53-450f-9fba-5a5f3089448a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:55.377261 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51df3ad-fa53-450f-9fba-5a5f3089448a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:55.377560 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51df3ad-fa53-450f-9fba-5a5f3089448a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:55.377798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51df3ad-fa53-450f-9fba-5a5f3089448a None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:55.378473 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51df3ad-fa53-450f-9fba-5a5f3089448a None admin] RBAC: Adding query filter params (name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:55.378775 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51df3ad-fa53-450f-9fba-5a5f3089448a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:55.380267 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51df3ad-fa53-450f-9fba-5a5f3089448a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:55.394016 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 273/557] 10.222.0.71 () {56 vars in 1103 bytes} [Fri Aug 26 08:37:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:55.394016 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51df3ad-fa53-450f-9fba-5a5f3089448a None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:55.424824 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb643fd8-8991-49d2-9861-63139cf43a19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:55.426257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb643fd8-8991-49d2-9861-63139cf43a19 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:55.443951 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:55.510352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51df3ad-fa53-450f-9fba-5a5f3089448a None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:55.512467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:55.558485 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 285/558] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:37:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:37:55.558779 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f6a4f-0b04-4df3-bcc5-0deb45bd0369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:55.558779 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f6a4f-0b04-4df3-bcc5-0deb45bd0369 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:55.558779 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f6a4f-0b04-4df3-bcc5-0deb45bd0369 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:55.558779 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f6a4f-0b04-4df3-bcc5-0deb45bd0369 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:55.559158 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518d8d9-1119-48fb-a75d-b038787c46c1 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:55.559158 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518d8d9-1119-48fb-a75d-b038787c46c1 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:55.559158 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518d8d9-1119-48fb-a75d-b038787c46c1 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:55.559158 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 286/559] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:55.559158 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518d8d9-1119-48fb-a75d-b038787c46c1 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:55.573480 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f6a4f-0b04-4df3-bcc5-0deb45bd0369 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:55.575077 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f6a4f-0b04-4df3-bcc5-0deb45bd0369 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:55.591600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-b73f6a4f-0b04-4df3-bcc5-0deb45bd0369 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:55.591600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:55.591600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:55.591600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:55.591600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:55.591600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:55.591600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:55.591600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:55.591600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:55.591600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:55.591600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:55.591600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:55.591600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:37:55.591600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:37:55.591600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:37:55.591600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:37:55.592612 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:55.592612 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:55.592612 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:55.592612 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:55.592612 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:55.592612 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:55.592612 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:55.592612 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:55.592612 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:55.592612 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:55.592612 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:55.592612 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:55.592612 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:55.592612 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:55.592612 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:55.592612 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:55.592612 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:37:55.593481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:37:55.593481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:55.593481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:55.593481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 274/560] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:55.593481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f6a4f-0b04-4df3-bcc5-0deb45bd0369 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:55.687773 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518d8d9-1119-48fb-a75d-b038787c46c1 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:55.713674 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:55.720695 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f6a4f-0b04-4df3-bcc5-0deb45bd0369 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:55.731482 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37579b-5e25-407d-91ad-1798e89550b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:55.732453 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37579b-5e25-407d-91ad-1798e89550b1 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:55.732737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37579b-5e25-407d-91ad-1798e89550b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:55.733013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37579b-5e25-407d-91ad-1798e89550b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:55.744807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37579b-5e25-407d-91ad-1798e89550b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:55.746373 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37579b-5e25-407d-91ad-1798e89550b1 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:55.758847 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e16543b-48d0-4532-bd10-d9fde7620732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:55.759513 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16543b-48d0-4532-bd10-d9fde7620732 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:55.759792 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16543b-48d0-4532-bd10-d9fde7620732 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:55.760061 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16543b-48d0-4532-bd10-d9fde7620732 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:55.762157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-6f37579b-5e25-407d-91ad-1798e89550b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:55.762157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:55.762157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:55.762157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:55.762157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:55.762157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:55.762157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:55.762157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:55.762157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:55.762157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:55.762157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:55.762157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:55.762157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:37:55.762157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:37:55.762157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:37:55.762157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:37:55.763232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:55.763232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:55.763232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:55.763232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:55.763232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:55.763232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:55.763232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:55.763232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:55.763232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:55.763232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:55.763232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:55.763232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:55.763232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:55.763232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:55.763232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:55.763232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:55.763232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:37:55.764157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:37:55.764157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:55.764157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:55.764157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 287/561] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:55.764712 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d9a11-3e9a-46fe-a2eb-a5c165a7ae3e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:55.764809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d9a11-3e9a-46fe-a2eb-a5c165a7ae3e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:55.765089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d9a11-3e9a-46fe-a2eb-a5c165a7ae3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:55.775842 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e16543b-48d0-4532-bd10-d9fde7620732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:55.777592 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e16543b-48d0-4532-bd10-d9fde7620732 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:55.783405 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-357d9a11-3e9a-46fe-a2eb-a5c165a7ae3e None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:55.784681 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-357d9a11-3e9a-46fe-a2eb-a5c165a7ae3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:55.793450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-9e16543b-48d0-4532-bd10-d9fde7620732 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:55.793450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:55.793450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:55.793450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:55.793450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:55.793450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:55.793450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:55.793450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:55.793450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:55.793450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:55.793450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:55.793450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:55.793450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:37:55.793450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:37:55.793450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:37:55.793450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:37:55.794496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:55.794496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:55.794496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:55.794496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:55.794496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:55.794496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:55.794496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:55.794496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:55.794496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:55.794496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:55.794496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:55.794496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:55.794496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:55.794496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:55.794496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:55.794496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:55.794496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:37:55.796328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:37:55.796328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:55.796328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:55.796328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 275/562] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:55.796328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e16543b-48d0-4532-bd10-d9fde7620732 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:55.803171 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:55.870638 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 288/563] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:55.871946 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdcf14-9820-4e0b-b2a7-e224b1e38c98 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:55.872213 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdcf14-9820-4e0b-b2a7-e224b1e38c98 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:55.872478 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdcf14-9820-4e0b-b2a7-e224b1e38c98 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:55.873780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 289/564] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:55.875045 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfdcf14-9820-4e0b-b2a7-e224b1e38c98 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:55.914413 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e16543b-48d0-4532-bd10-d9fde7620732 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:55.959380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4b9bb-2c15-438d-8b16-7020224b65db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:55.960047 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4b9bb-2c15-438d-8b16-7020224b65db None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:55.960163 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4b9bb-2c15-438d-8b16-7020224b65db None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:55.960411 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4b9bb-2c15-438d-8b16-7020224b65db None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:55.961080 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c4b9bb-2c15-438d-8b16-7020224b65db None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:55.961382 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c4b9bb-2c15-438d-8b16-7020224b65db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:55.962832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c4b9bb-2c15-438d-8b16-7020224b65db None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:55.985033 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 276/565] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:55.985033 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4b9bb-2c15-438d-8b16-7020224b65db None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:55.996948 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:56.041525 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfdcf14-9820-4e0b-b2a7-e224b1e38c98 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:56.082732 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65bc2a-c960-4c35-8ce1-6dfd15b515d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:56.083414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65bc2a-c960-4c35-8ce1-6dfd15b515d4 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:56.083681 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65bc2a-c960-4c35-8ce1-6dfd15b515d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:56.083944 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65bc2a-c960-4c35-8ce1-6dfd15b515d4 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:56.085036 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65bc2a-c960-4c35-8ce1-6dfd15b515d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:56.085357 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65bc2a-c960-4c35-8ce1-6dfd15b515d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:56.087150 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65bc2a-c960-4c35-8ce1-6dfd15b515d4 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:56.107113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 290/566] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:56.107113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f64348-45af-4aef-8653-7c32d7c839bd None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:56.107113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f64348-45af-4aef-8653-7c32d7c839bd None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:56.107113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f64348-45af-4aef-8653-7c32d7c839bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:56.120469 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4b9bb-2c15-438d-8b16-7020224b65db None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:56.131341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-79f64348-45af-4aef-8653-7c32d7c839bd None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:56.132571 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-79f64348-45af-4aef-8653-7c32d7c839bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:56.141812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:56.165068 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143131f8-496e-4cbb-a467-d5f0932ecba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:56.165841 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143131f8-496e-4cbb-a467-d5f0932ecba6 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:56.166249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143131f8-496e-4cbb-a467-d5f0932ecba6 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:56.166646 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143131f8-496e-4cbb-a467-d5f0932ecba6 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:56.167588 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143131f8-496e-4cbb-a467-d5f0932ecba6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:56.167983 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143131f8-496e-4cbb-a467-d5f0932ecba6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:56.169641 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143131f8-496e-4cbb-a467-d5f0932ecba6 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:56.181711 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 277/567] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:56.182592 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143131f8-496e-4cbb-a467-d5f0932ecba6 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:56.210315 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 291/568] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:56.211115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f64348-45af-4aef-8653-7c32d7c839bd None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:56.288072 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:56.332472 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143131f8-496e-4cbb-a467-d5f0932ecba6 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:56.344700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f64348-45af-4aef-8653-7c32d7c839bd None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:56.387111 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ce51b-ff17-4e6b-9e4b-34e9df30e9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:56.387997 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ce51b-ff17-4e6b-9e4b-34e9df30e9fd None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:56.388436 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ce51b-ff17-4e6b-9e4b-34e9df30e9fd None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:56.388868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ce51b-ff17-4e6b-9e4b-34e9df30e9fd None admin] PATH_INFO: `/v3/users/nova` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:56.397798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ce51b-ff17-4e6b-9e4b-34e9df30e9fd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:56.399532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817ce51b-ff17-4e6b-9e4b-34e9df30e9fd None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:56.405398 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374185e-450d-4a8d-8429-1aa80eaf18dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:56.406101 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374185e-450d-4a8d-8429-1aa80eaf18dc None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:56.406382 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374185e-450d-4a8d-8429-1aa80eaf18dc None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:56.406651 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374185e-450d-4a8d-8429-1aa80eaf18dc None admin] PATH_INFO: `/v3/users/glance` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:56.414928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b374185e-450d-4a8d-8429-1aa80eaf18dc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:56.415509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-817ce51b-ff17-4e6b-9e4b-34e9df30e9fd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 08:37:56.415509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:56.415509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:56.415509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:56.415509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:56.415509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:56.415509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:56.415509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:56.415509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:56.415509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:56.415509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:56.415509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:56.415509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:37:56.415509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:37:56.415509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:37:56.415509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:37:56.416412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:56.416412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:56.416412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:37:56.416412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:37:56.416412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:37:56.416412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:37:56.416412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:56.416412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:56.416412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:56.416412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:56.416412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:56.416412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:56.416412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:56.416412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:56.416412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:56.416412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:56.416412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:56.416412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:56.417337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:56.417337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:56.417337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:37:56.417337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:37:56.417337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:37:56.417337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:37:56.417337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:37:56.417337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:37:56.417337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 08:37:56.417337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:56.417859 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b374185e-450d-4a8d-8429-1aa80eaf18dc None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:56.419194 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 278/569] 10.222.0.71 () {56 vars in 1093 bytes} [Fri Aug 26 08:37:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:56.420703 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ce51b-ff17-4e6b-9e4b-34e9df30e9fd None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:56.436631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-b374185e-450d-4a8d-8429-1aa80eaf18dc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 26 08:37:56.436631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:56.436631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:56.436631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:56.436631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:56.436631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:56.436631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:56.436631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:56.436631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:56.436631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:56.436631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:56.436631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:56.436631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:37:56.436631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:37:56.436631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:37:56.436631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:37:56.442812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:56.442812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:56.442812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:37:56.442812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:37:56.442812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:37:56.442812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:37:56.442812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:56.442812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:56.442812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:56.442812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:56.442812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:56.442812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:56.442812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:56.442812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:56.442812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:56.442812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:56.442812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:56.442812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:56.444020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:56.444020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:56.444020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:37:56.444020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:37:56.444020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:37:56.444020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:37:56.444020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:37:56.444020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:37:56.444020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 26 08:37:56.444020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:56.444020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 292/570] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:37:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:56.444020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374185e-450d-4a8d-8429-1aa80eaf18dc None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:56.445878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:56.542401 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817ce51b-ff17-4e6b-9e4b-34e9df30e9fd None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:56.555903 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374185e-450d-4a8d-8429-1aa80eaf18dc None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:56.572022 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:56.581097 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff883ae4-e147-4c40-b79c-e7a8620f65e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:56.581840 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff883ae4-e147-4c40-b79c-e7a8620f65e9 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:56.582115 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff883ae4-e147-4c40-b79c-e7a8620f65e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:56.582388 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff883ae4-e147-4c40-b79c-e7a8620f65e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:56.594108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff883ae4-e147-4c40-b79c-e7a8620f65e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:56.594338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3200a1e-89a6-48c0-aa27-4bb759d01c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:56.595017 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3200a1e-89a6-48c0-aa27-4bb759d01c96 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:56.595300 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3200a1e-89a6-48c0-aa27-4bb759d01c96 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:56.595572 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3200a1e-89a6-48c0-aa27-4bb759d01c96 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:56.599430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-ff883ae4-e147-4c40-b79c-e7a8620f65e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:37:56.599430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:56.599430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:56.599430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:56.599430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:56.599430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:56.599430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:56.599430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:56.599430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:56.599430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:56.599430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:56.599430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:56.599430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:56.599430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:56.599430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:56.599430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:56.600557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:56.600557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:56.600557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:56.600557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:56.600557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:56.600557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:56.600557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:56.600557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:56.600557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:56.600557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:56.600557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:56.600557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:56.600557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:56.600557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:56.600557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:56.600557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:56.601427 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:56.601427 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:56.601427 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:56.601427 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:56.601427 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:56.601427 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:56.601427 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:56.601427 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:56.601427 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:56.601427 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:56.601427 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:37:56.601427 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:56.601427 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 279/571] 10.222.0.71 () {56 vars in 1102 bytes} [Fri Aug 26 08:37:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:56.602316 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff883ae4-e147-4c40-b79c-e7a8620f65e9 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:56.609025 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3200a1e-89a6-48c0-aa27-4bb759d01c96 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:56.610576 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3200a1e-89a6-48c0-aa27-4bb759d01c96 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:56.623362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-b3200a1e-89a6-48c0-aa27-4bb759d01c96 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:37:56.623362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:56.623362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:56.623362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:56.623362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:56.623362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:56.623362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:56.623362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:56.623362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:56.623362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:56.623362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:56.623362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:56.623362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:37:56.623362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:37:56.623362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:37:56.624587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:37:56.624587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:56.624587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:56.624587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:56.624587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:56.624587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:56.624587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:56.624587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:56.624587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:56.624587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:56.624587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:56.624587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:56.624587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:56.624587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:56.624587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:56.624587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:56.624587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:56.625860 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:37:56.625860 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:37:56.625860 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:37:56.625860 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:37:56.625860 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:37:56.625860 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:37:56.625860 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:37:56.625860 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:56.625860 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 293/572] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:37:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:56.625860 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3200a1e-89a6-48c0-aa27-4bb759d01c96 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:56.702249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:56.726540 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff883ae4-e147-4c40-b79c-e7a8620f65e9 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:56.754868 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3200a1e-89a6-48c0-aa27-4bb759d01c96 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:56.773852 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a37327-420d-46a7-bd68-c2fe65ab7560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:56.774773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a37327-420d-46a7-bd68-c2fe65ab7560 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:56.775100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a37327-420d-46a7-bd68-c2fe65ab7560 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:56.775381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a37327-420d-46a7-bd68-c2fe65ab7560 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:56.776206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a37327-420d-46a7-bd68-c2fe65ab7560 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:56.776516 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a37327-420d-46a7-bd68-c2fe65ab7560 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:56.778030 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a37327-420d-46a7-bd68-c2fe65ab7560 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:56.800496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 280/573] 10.222.0.71 () {56 vars in 1133 bytes} [Fri Aug 26 08:37:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:56.801767 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d556a-4b3f-45c3-81c5-400d627f3645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:56.801977 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a37327-420d-46a7-bd68-c2fe65ab7560 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:56.802467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d556a-4b3f-45c3-81c5-400d627f3645 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:56.802751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d556a-4b3f-45c3-81c5-400d627f3645 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:56.803059 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d556a-4b3f-45c3-81c5-400d627f3645 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:56.803823 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d556a-4b3f-45c3-81c5-400d627f3645 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:56.804137 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d556a-4b3f-45c3-81c5-400d627f3645 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:56.805593 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d556a-4b3f-45c3-81c5-400d627f3645 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:56.828559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 294/574] 10.222.0.71 () {56 vars in 1137 bytes} [Fri Aug 26 08:37:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:56.829418 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d556a-4b3f-45c3-81c5-400d627f3645 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:56.860726 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:56.929387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a37327-420d-46a7-bd68-c2fe65ab7560 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:56.945684 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d556a-4b3f-45c3-81c5-400d627f3645 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:56.967329 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c115e-56f2-401d-80ce-67f5ad35f5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:56.967968 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c115e-56f2-401d-80ce-67f5ad35f5b8 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:56.968237 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c115e-56f2-401d-80ce-67f5ad35f5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:56.968428 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:56.968559 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c115e-56f2-401d-80ce-67f5ad35f5b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:56.980393 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c115e-56f2-401d-80ce-67f5ad35f5b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:37:56.985872 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-795c115e-56f2-401d-80ce-67f5ad35f5b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:37:56.985872 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:56.985872 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:56.985872 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:56.985872 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:56.985872 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:56.985872 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:56.985872 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:56.985872 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:56.985872 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:56.985872 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:56.985872 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:56.985872 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:37:56.985872 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:37:56.985872 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:37:56.985872 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:37:56.987032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:37:56.987032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:37:56.987032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:37:56.987032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:37:56.987032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:56.987032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:56.987032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:56.987032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:56.987032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:56.987032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:56.987032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:56.987032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:56.987032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:56.987032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:56.987032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:56.987032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:56.987881 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:56.987881 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:56.987881 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:56.987881 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:56.987881 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:37:56.987881 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:37:56.987881 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:37:56.987881 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:37:56.987881 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:37:56.987881 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:37:56.987881 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:37:56.987881 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:56.987881 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 281/575] 10.222.0.71 () {56 vars in 1102 bytes} [Fri Aug 26 08:37:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:56.987881 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c115e-56f2-401d-80ce-67f5ad35f5b8 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:56.993069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc00c94d-9346-4888-9101-1dedc737cd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:56.993710 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00c94d-9346-4888-9101-1dedc737cd14 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:56.993983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00c94d-9346-4888-9101-1dedc737cd14 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:56.994243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00c94d-9346-4888-9101-1dedc737cd14 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:56.994962 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc00c94d-9346-4888-9101-1dedc737cd14 None admin] RBAC: Adding query filter params (name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:56.995269 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc00c94d-9346-4888-9101-1dedc737cd14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:56.996657 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc00c94d-9346-4888-9101-1dedc737cd14 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:57.006196 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 295/576] 10.222.0.71 () {56 vars in 1103 bytes} [Fri Aug 26 08:37:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:57.007139 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc00c94d-9346-4888-9101-1dedc737cd14 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:57.132773 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc00c94d-9346-4888-9101-1dedc737cd14 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:57.136691 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c115e-56f2-401d-80ce-67f5ad35f5b8 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:57.144379 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:57.168433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aba2c9-cb92-4a4f-862d-2524be888afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:57.169067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aba2c9-cb92-4a4f-862d-2524be888afe None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:57.169345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aba2c9-cb92-4a4f-862d-2524be888afe None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:57.169613 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aba2c9-cb92-4a4f-862d-2524be888afe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:57.172725 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b20b3-edc9-4d3e-a346-72c15466dcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:57.173301 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b20b3-edc9-4d3e-a346-72c15466dcdd None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:57.173573 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b20b3-edc9-4d3e-a346-72c15466dcdd None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:57.173853 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b20b3-edc9-4d3e-a346-72c15466dcdd None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:57.174544 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b20b3-edc9-4d3e-a346-72c15466dcdd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:57.174843 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b20b3-edc9-4d3e-a346-72c15466dcdd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:57.176347 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b20b3-edc9-4d3e-a346-72c15466dcdd None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:57.180461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aba2c9-cb92-4a4f-862d-2524be888afe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:57.181928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aba2c9-cb92-4a4f-862d-2524be888afe None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:57.188221 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 282/577] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:37:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:57.189080 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b20b3-edc9-4d3e-a346-72c15466dcdd None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:57.196809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-c3aba2c9-cb92-4a4f-862d-2524be888afe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:57.196809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:57.196809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:57.196809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:57.196809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:57.196809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:57.196809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:57.196809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:57.196809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:57.196809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:57.196809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:57.196809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:57.196809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:37:57.196809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:37:57.196809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:37:57.196809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:37:57.197842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:57.197842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:57.197842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:57.197842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:57.197842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:57.197842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:57.197842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:57.197842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:57.197842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:57.197842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:57.197842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:57.197842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:57.197842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:57.197842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:57.197842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:57.197842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:57.197842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:37:57.198730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:37:57.198730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:57.198730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:57.202689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 296/578] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:57.203561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aba2c9-cb92-4a4f-862d-2524be888afe None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:57.240028 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:57.307721 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b20b3-edc9-4d3e-a346-72c15466dcdd None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:57.325528 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aba2c9-cb92-4a4f-862d-2524be888afe None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:57.352750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce570635-4d4a-4968-bea0-435e6d634e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:57.353493 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce570635-4d4a-4968-bea0-435e6d634e0b None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:57.353923 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce570635-4d4a-4968-bea0-435e6d634e0b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:57.354031 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce570635-4d4a-4968-bea0-435e6d634e0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:57.365710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce570635-4d4a-4968-bea0-435e6d634e0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:57.367536 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce570635-4d4a-4968-bea0-435e6d634e0b None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:57.369890 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e75998-46a9-47f1-a332-f4c1325a9714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:57.370786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e75998-46a9-47f1-a332-f4c1325a9714 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:57.371240 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e75998-46a9-47f1-a332-f4c1325a9714 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:57.371642 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e75998-46a9-47f1-a332-f4c1325a9714 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:57.372664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e75998-46a9-47f1-a332-f4c1325a9714 None admin] RBAC: Adding query filter params (name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:57.373123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e75998-46a9-47f1-a332-f4c1325a9714 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:57.375298 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e75998-46a9-47f1-a332-f4c1325a9714 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:57.386154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-ce570635-4d4a-4968-bea0-435e6d634e0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:57.386154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:57.386154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:57.386154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:57.386154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:57.386154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:57.386154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:57.386154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:57.386154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:57.386154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:57.386154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:57.386154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:57.386154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:37:57.386154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:37:57.386154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:37:57.386154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:37:57.387428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:57.387428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:57.387428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:57.387428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:57.387428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:57.387428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:57.387428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:57.387428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:57.387428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:57.387428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:57.387428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:57.387428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:57.387428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:57.387428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:57.387428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:57.387428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:57.387428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:37:57.388313 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:37:57.388313 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:57.388313 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:57.388313 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 283/579] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:57.388313 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce570635-4d4a-4968-bea0-435e6d634e0b None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:57.391268 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 297/580] 10.222.0.71 () {56 vars in 1103 bytes} [Fri Aug 26 08:37:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:57.393086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e75998-46a9-47f1-a332-f4c1325a9714 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:57.415235 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:57.507383 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:57.511437 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce570635-4d4a-4968-bea0-435e6d634e0b None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:57.515232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e75998-46a9-47f1-a332-f4c1325a9714 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:57.550290 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98b2f1-4a93-4f2e-90e0-ef54c4d43209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:57.550944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98b2f1-4a93-4f2e-90e0-ef54c4d43209 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:57.551223 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98b2f1-4a93-4f2e-90e0-ef54c4d43209 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:57.551519 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98b2f1-4a93-4f2e-90e0-ef54c4d43209 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:57.560854 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d83d17-b0ce-4ed9-9337-c38cf1766971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:57.561441 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d83d17-b0ce-4ed9-9337-c38cf1766971 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:57.561716 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d83d17-b0ce-4ed9-9337-c38cf1766971 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:57.562002 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d83d17-b0ce-4ed9-9337-c38cf1766971 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:57.562739 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d83d17-b0ce-4ed9-9337-c38cf1766971 None admin] RBAC: Adding query filter params (role.id=33ac296b51c143e7bb44211175752edb, scope.project.id=543d44293d894a7fbb83401f7a26dd9b, user.id=1723e961375b45b7b3bd99ddeb69683f) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:57.563077 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d83d17-b0ce-4ed9-9337-c38cf1766971 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:57.563746 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98b2f1-4a93-4f2e-90e0-ef54c4d43209 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:57.564514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d83d17-b0ce-4ed9-9337-c38cf1766971 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:57.565186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98b2f1-4a93-4f2e-90e0-ef54c4d43209 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:57.571010 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 298/581] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:37:57 2022] GET /identity/v3/role_assignments?user.id=1723e961375b45b7b3bd99ddeb69683f&scope.project.id=543d44293d894a7fbb83401f7a26dd9b&role.id=33ac296b51c143e7bb44211175752edb => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:57.571840 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d83d17-b0ce-4ed9-9337-c38cf1766971 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:57.581426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-6e98b2f1-4a93-4f2e-90e0-ef54c4d43209 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:57.581426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:57.581426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:57.581426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:57.581426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:57.581426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:57.581426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:57.581426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:57.581426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:57.581426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:57.581426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:57.581426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:57.581426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:37:57.581426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:37:57.581426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:37:57.581426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:37:57.582699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:57.582699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:57.582699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:57.582699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:57.582699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:57.582699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:57.582699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:57.582699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:57.582699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:57.582699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:57.582699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:57.582699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:57.582699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:57.582699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:57.582699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:57.582699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:57.582699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:37:57.583641 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:37:57.583641 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:57.583641 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:57.583641 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 284/582] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:57.583965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98b2f1-4a93-4f2e-90e0-ef54c4d43209 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:57.679176 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:57.731497 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d83d17-b0ce-4ed9-9337-c38cf1766971 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:57.732053 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98b2f1-4a93-4f2e-90e0-ef54c4d43209 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:57.775585 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b69ba-8dbf-4836-b414-6907649baece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:57.776945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b69ba-8dbf-4836-b414-6907649baece None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:57.777453 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b69ba-8dbf-4836-b414-6907649baece None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:57.777924 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b69ba-8dbf-4836-b414-6907649baece None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:57.778843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b69ba-8dbf-4836-b414-6907649baece None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:57.779395 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b69ba-8dbf-4836-b414-6907649baece None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:57.781020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b69ba-8dbf-4836-b414-6907649baece None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:57.796118 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227b2b0-a665-4fbe-80e5-9d1a13a78f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:57.796763 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227b2b0-a665-4fbe-80e5-9d1a13a78f86 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:57.797043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227b2b0-a665-4fbe-80e5-9d1a13a78f86 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:57.797316 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227b2b0-a665-4fbe-80e5-9d1a13a78f86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:57.808575 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 299/583] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:57.809618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6227b2b0-a665-4fbe-80e5-9d1a13a78f86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:57.810099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b69ba-8dbf-4836-b414-6907649baece None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:57.811316 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6227b2b0-a665-4fbe-80e5-9d1a13a78f86 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:57.838330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-6227b2b0-a665-4fbe-80e5-9d1a13a78f86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:57.838330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:57.838330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:57.838330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:57.838330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:57.838330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:57.838330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:57.838330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:57.838330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:57.838330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:57.838330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:57.838330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:57.838330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:37:57.838330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:37:57.838330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:37:57.838330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:37:57.840396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:57.840396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:57.840396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:57.840396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:57.840396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:57.840396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:57.840396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:57.840396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:57.840396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:57.840396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:57.840396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:57.840396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:57.840396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:57.840396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:57.840396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:57.840396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:57.840396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:37:57.842201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:37:57.842201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:57.842201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:57.842201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 285/584] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:57.842201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227b2b0-a665-4fbe-80e5-9d1a13a78f86 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:57.852488 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:57.968946 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b69ba-8dbf-4836-b414-6907649baece None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:57.971298 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227b2b0-a665-4fbe-80e5-9d1a13a78f86 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:57.999051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:58.013036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e487ef-dbea-48bf-b750-55c1367d8cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:58.013249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcf1a3-7d50-4b63-a3e7-76f2bf0fdec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:58.013672 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e487ef-dbea-48bf-b750-55c1367d8cd5 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:58.013781 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcf1a3-7d50-4b63-a3e7-76f2bf0fdec5 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:58.013993 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e487ef-dbea-48bf-b750-55c1367d8cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:58.014088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcf1a3-7d50-4b63-a3e7-76f2bf0fdec5 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:58.014240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e487ef-dbea-48bf-b750-55c1367d8cd5 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:58.014330 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcf1a3-7d50-4b63-a3e7-76f2bf0fdec5 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:58.014866 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e487ef-dbea-48bf-b750-55c1367d8cd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:58.015075 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcf1a3-7d50-4b63-a3e7-76f2bf0fdec5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:58.015190 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e487ef-dbea-48bf-b750-55c1367d8cd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:58.015362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcf1a3-7d50-4b63-a3e7-76f2bf0fdec5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:58.016550 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e487ef-dbea-48bf-b750-55c1367d8cd5 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:58.016711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bcf1a3-7d50-4b63-a3e7-76f2bf0fdec5 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:58.030255 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 286/585] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:58.039451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e487ef-dbea-48bf-b750-55c1367d8cd5 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:58.039646 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 300/586] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:58.039646 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcf1a3-7d50-4b63-a3e7-76f2bf0fdec5 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:58.131166 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:58.155052 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcf1a3-7d50-4b63-a3e7-76f2bf0fdec5 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:58.178596 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e487ef-dbea-48bf-b750-55c1367d8cd5 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:58.204704 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb46f4a-8ea4-49f4-86c0-92fc22c50cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:58.205431 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb46f4a-8ea4-49f4-86c0-92fc22c50cf8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:58.205623 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb46f4a-8ea4-49f4-86c0-92fc22c50cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:58.205898 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb46f4a-8ea4-49f4-86c0-92fc22c50cf8 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:58.206555 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb46f4a-8ea4-49f4-86c0-92fc22c50cf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:58.206865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb46f4a-8ea4-49f4-86c0-92fc22c50cf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:58.208377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb46f4a-8ea4-49f4-86c0-92fc22c50cf8 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:58.227724 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 301/587] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:58.227724 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb46f4a-8ea4-49f4-86c0-92fc22c50cf8 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:58.234408 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911b796-6c35-4ecd-a866-62a76bfc8cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:58.235105 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911b796-6c35-4ecd-a866-62a76bfc8cc5 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:58.235458 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911b796-6c35-4ecd-a866-62a76bfc8cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:58.235736 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911b796-6c35-4ecd-a866-62a76bfc8cc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:58.243810 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911b796-6c35-4ecd-a866-62a76bfc8cc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:58.245386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911b796-6c35-4ecd-a866-62a76bfc8cc5 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:58.257985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-8911b796-6c35-4ecd-a866-62a76bfc8cc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:37:58.257985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:58.257985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:58.257985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:58.257985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:58.257985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:58.257985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:58.257985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:58.257985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:58.257985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:58.257985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:58.257985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:58.257985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:37:58.257985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:37:58.257985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:37:58.258959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:37:58.258959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:58.258959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:58.258959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:58.258959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:58.258959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:58.258959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:58.258959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:58.258959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:58.258959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:58.258959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:58.258959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:58.258959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:58.258959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:58.258959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:58.258959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:58.258959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:58.259882 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:37:58.259882 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:37:58.259882 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:37:58.259882 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:37:58.259882 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:37:58.259882 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:37:58.259882 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:37:58.259882 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:58.259882 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 287/588] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:37:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:58.260369 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911b796-6c35-4ecd-a866-62a76bfc8cc5 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:58.327735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:58.349918 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb46f4a-8ea4-49f4-86c0-92fc22c50cf8 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:58.397395 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775807a-6741-44de-a98b-52f454e2a54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:58.398149 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775807a-6741-44de-a98b-52f454e2a54c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:58.398517 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775807a-6741-44de-a98b-52f454e2a54c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:58.398855 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775807a-6741-44de-a98b-52f454e2a54c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:58.399917 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911b796-6c35-4ecd-a866-62a76bfc8cc5 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:58.406712 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0775807a-6741-44de-a98b-52f454e2a54c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:58.408332 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0775807a-6741-44de-a98b-52f454e2a54c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:58.411815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:58.423461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-0775807a-6741-44de-a98b-52f454e2a54c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:37:58.423461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:58.423461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:58.423461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:58.423461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:58.423461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:58.423461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:58.423461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:58.423461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:58.423461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:58.423461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:58.423461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:58.423461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:37:58.423461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:37:58.423461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:37:58.424809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:37:58.424809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:58.424809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:58.424809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:58.424809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:58.424809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:58.424809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:58.424809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:58.424809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:58.424809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:58.424809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:58.424809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:58.424809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:58.424809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:58.424809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:58.424809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:58.424809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:58.427256 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:37:58.427256 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:37:58.427256 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:37:58.427256 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:37:58.427256 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:37:58.427256 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:37:58.427256 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:37:58.427256 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:58.427256 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 302/589] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:37:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:58.427256 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775807a-6741-44de-a98b-52f454e2a54c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:58.455293 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb712d-6987-4839-8b28-3b5aff9f434c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:58.456076 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb712d-6987-4839-8b28-3b5aff9f434c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:58.456377 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb712d-6987-4839-8b28-3b5aff9f434c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:58.456733 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb712d-6987-4839-8b28-3b5aff9f434c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:58.466582 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bb712d-6987-4839-8b28-3b5aff9f434c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:58.468093 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bb712d-6987-4839-8b28-3b5aff9f434c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:58.483015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-24bb712d-6987-4839-8b28-3b5aff9f434c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 08:37:58.483015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:58.483015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:58.483015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:58.483015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:58.483015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:58.483015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:58.483015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:58.483015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:58.483015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:58.483015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:58.483015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:58.483015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:37:58.483015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:37:58.483015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:37:58.483015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:37:58.484349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:58.484349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:58.484349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:37:58.484349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:37:58.484349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:37:58.484349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:37:58.484349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:58.484349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:58.484349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:58.484349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:58.484349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:58.484349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:58.484349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:58.484349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:58.484349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:58.484349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:58.484349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:58.484349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:58.485309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:58.485309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:58.485309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:37:58.485309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:37:58.485309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:37:58.485309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:37:58.485309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:37:58.485309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:37:58.485309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 08:37:58.485309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:58.485309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 288/590] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:37:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:58.485309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb712d-6987-4839-8b28-3b5aff9f434c None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:58.556756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775807a-6741-44de-a98b-52f454e2a54c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:58.604345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5cd15a-f13f-498e-8088-f2f976a0760e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:58.604989 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5cd15a-f13f-498e-8088-f2f976a0760e None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:58.605265 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5cd15a-f13f-498e-8088-f2f976a0760e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:58.605536 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5cd15a-f13f-498e-8088-f2f976a0760e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:58.615978 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5cd15a-f13f-498e-8088-f2f976a0760e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:58.617767 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5cd15a-f13f-498e-8088-f2f976a0760e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:58.623078 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb712d-6987-4839-8b28-3b5aff9f434c None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:58.625215 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:58.632490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-7f5cd15a-f13f-498e-8088-f2f976a0760e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 08:37:58.632490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:58.632490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:58.632490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:58.632490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:58.632490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:58.632490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:58.632490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:58.632490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:58.632490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:58.632490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:58.632490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:58.632490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:37:58.632490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:37:58.632490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:37:58.632490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:37:58.634168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:58.634168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:58.634168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:37:58.634168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:37:58.634168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:37:58.634168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:37:58.634168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:58.634168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:58.634168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:58.634168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:58.634168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:58.634168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:58.634168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:58.634168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:58.634168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:58.634168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:58.634168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:58.634168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:58.636470 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:58.636470 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:58.636470 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:37:58.636470 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:37:58.636470 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:37:58.636470 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:37:58.636470 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:37:58.636470 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:37:58.636470 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 08:37:58.636470 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:58.636470 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 303/591] 10.222.0.71 () {56 vars in 1102 bytes} [Fri Aug 26 08:37:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:58.636470 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5cd15a-f13f-498e-8088-f2f976a0760e None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:58.681259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e4920-8d09-4d1e-8c71-8f296f023ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:58.682254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e4920-8d09-4d1e-8c71-8f296f023ce5 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:58.682708 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e4920-8d09-4d1e-8c71-8f296f023ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:58.683171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e4920-8d09-4d1e-8c71-8f296f023ce5 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:58.691841 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e4920-8d09-4d1e-8c71-8f296f023ce5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:58.691841 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e4920-8d09-4d1e-8c71-8f296f023ce5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:58.691841 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e4920-8d09-4d1e-8c71-8f296f023ce5 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:58.700786 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 289/592] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:37:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:58.701903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e4920-8d09-4d1e-8c71-8f296f023ce5 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:58.722759 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:58.775153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5cd15a-f13f-498e-8088-f2f976a0760e None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:58.812117 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a26ac9-9e2b-48c7-b92a-ea4b2baf2aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:58.812798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e4920-8d09-4d1e-8c71-8f296f023ce5 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:58.812886 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a26ac9-9e2b-48c7-b92a-ea4b2baf2aba None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:58.813169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a26ac9-9e2b-48c7-b92a-ea4b2baf2aba None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:58.813332 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a26ac9-9e2b-48c7-b92a-ea4b2baf2aba None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:58.814042 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a26ac9-9e2b-48c7-b92a-ea4b2baf2aba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:58.814351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a26ac9-9e2b-48c7-b92a-ea4b2baf2aba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:58.815976 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a26ac9-9e2b-48c7-b92a-ea4b2baf2aba None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:58.836581 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 304/593] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:37:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:58.836581 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a26ac9-9e2b-48c7-b92a-ea4b2baf2aba None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:58.861423 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e64470-e7d3-4296-902a-0cce014a08d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:58.862029 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e64470-e7d3-4296-902a-0cce014a08d8 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:58.862306 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e64470-e7d3-4296-902a-0cce014a08d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:58.862572 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e64470-e7d3-4296-902a-0cce014a08d8 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:58.863343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e64470-e7d3-4296-902a-0cce014a08d8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:58.863646 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e64470-e7d3-4296-902a-0cce014a08d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:58.865225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e64470-e7d3-4296-902a-0cce014a08d8 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:58.885123 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 290/594] 10.222.0.71 () {56 vars in 1137 bytes} [Fri Aug 26 08:37:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:58.885802 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e64470-e7d3-4296-902a-0cce014a08d8 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:58.912372 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:58.984518 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a26ac9-9e2b-48c7-b92a-ea4b2baf2aba None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:59.007480 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e64470-e7d3-4296-902a-0cce014a08d8 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:59.012085 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:59.029418 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337cd0d-db6b-4d26-85c3-5e37b0700f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:59.029949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337cd0d-db6b-4d26-85c3-5e37b0700f61 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:59.030225 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337cd0d-db6b-4d26-85c3-5e37b0700f61 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:59.030497 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337cd0d-db6b-4d26-85c3-5e37b0700f61 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:59.031279 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3337cd0d-db6b-4d26-85c3-5e37b0700f61 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:59.031586 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3337cd0d-db6b-4d26-85c3-5e37b0700f61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:59.033018 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3337cd0d-db6b-4d26-85c3-5e37b0700f61 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:59.048698 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917aa40-8d6b-4bdd-aa8e-cff83b7aa2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:59.049470 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917aa40-8d6b-4bdd-aa8e-cff83b7aa2db None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:59.049749 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917aa40-8d6b-4bdd-aa8e-cff83b7aa2db None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:59.050016 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917aa40-8d6b-4bdd-aa8e-cff83b7aa2db None admin] PATH_INFO: `/v3/projects/543d44293d894a7fbb83401f7a26dd9b/users/f16d1768715a41648af3fe6afadd7477/roles/33ac296b51c143e7bb44211175752edb` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:59.053278 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 305/595] 10.222.0.71 () {56 vars in 1139 bytes} [Fri Aug 26 08:37:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:59.058510 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337cd0d-db6b-4d26-85c3-5e37b0700f61 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:59.071993 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917aa40-8d6b-4bdd-aa8e-cff83b7aa2db None admin] RBAC: Authorizing `identity:create_grant(project_id=543d44293d894a7fbb83401f7a26dd9b, user_id=f16d1768715a41648af3fe6afadd7477, role_id=33ac296b51c143e7bb44211175752edb)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:59.071993 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917aa40-8d6b-4bdd-aa8e-cff83b7aa2db None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:59.098928 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 291/596] 10.222.0.71 () {58 vars in 1439 bytes} [Fri Aug 26 08:37:58 2022] PUT /identity/v3/projects/543d44293d894a7fbb83401f7a26dd9b/users/f16d1768715a41648af3fe6afadd7477/roles/33ac296b51c143e7bb44211175752edb => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:37:59.098928 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917aa40-8d6b-4bdd-aa8e-cff83b7aa2db None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:59.219606 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337cd0d-db6b-4d26-85c3-5e37b0700f61 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:59.228621 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:59.260303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e3f07-c2ff-4960-9461-e9027e6e0939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:59.261323 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e3f07-c2ff-4960-9461-e9027e6e0939 None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:59.261789 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e3f07-c2ff-4960-9461-e9027e6e0939 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:59.262285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e3f07-c2ff-4960-9461-e9027e6e0939 None admin] PATH_INFO: `/v3/projects/543d44293d894a7fbb83401f7a26dd9b/users/0d178c717a3e49319545e70cb58cbbed/roles/33ac296b51c143e7bb44211175752edb` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:59.273687 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917aa40-8d6b-4bdd-aa8e-cff83b7aa2db None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:59.279521 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e3f07-c2ff-4960-9461-e9027e6e0939 None admin] RBAC: Authorizing `identity:create_grant(project_id=543d44293d894a7fbb83401f7a26dd9b, user_id=0d178c717a3e49319545e70cb58cbbed, role_id=33ac296b51c143e7bb44211175752edb)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:59.281023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e3f07-c2ff-4960-9461-e9027e6e0939 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:59.299923 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 306/597] 10.222.0.71 () {58 vars in 1439 bytes} [Fri Aug 26 08:37:59 2022] PUT /identity/v3/projects/543d44293d894a7fbb83401f7a26dd9b/users/0d178c717a3e49319545e70cb58cbbed/roles/33ac296b51c143e7bb44211175752edb => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:37:59.300804 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e3f07-c2ff-4960-9461-e9027e6e0939 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:59.333268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb9999-c7b0-41dd-b0f1-4db7986e8387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:59.335793 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb9999-c7b0-41dd-b0f1-4db7986e8387 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:59.336838 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb9999-c7b0-41dd-b0f1-4db7986e8387 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:59.337611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb9999-c7b0-41dd-b0f1-4db7986e8387 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:59.373288 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb9999-c7b0-41dd-b0f1-4db7986e8387 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:59.376099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb9999-c7b0-41dd-b0f1-4db7986e8387 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:59.384293 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:59.418058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-41eb9999-c7b0-41dd-b0f1-4db7986e8387 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:59.418058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:59.418058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:59.418058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:59.418058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:59.418058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:59.418058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:59.418058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:59.418058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:59.418058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:59.418058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:59.418058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:59.418058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:37:59.418058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:37:59.418058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:37:59.418058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:37:59.420008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:59.420008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:59.420008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:59.420008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:59.420008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:59.420008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:59.420008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:59.420008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:59.420008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:59.420008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:59.420008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:59.420008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:59.420008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:59.420008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:59.420008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:59.420008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:59.420008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:37:59.421849 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:37:59.421849 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:59.421849 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:37:59.424986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 292/598] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:59.433211 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb9999-c7b0-41dd-b0f1-4db7986e8387 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:59.550257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e3f07-c2ff-4960-9461-e9027e6e0939 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:59.603566 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d58c6dd-637f-44f6-85d2-c3f9065272a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:59.603811 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb9999-c7b0-41dd-b0f1-4db7986e8387 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:59.604342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d58c6dd-637f-44f6-85d2-c3f9065272a6 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:59.604625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d58c6dd-637f-44f6-85d2-c3f9065272a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:59.604901 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d58c6dd-637f-44f6-85d2-c3f9065272a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:59.620243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d58c6dd-637f-44f6-85d2-c3f9065272a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:59.622505 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d58c6dd-637f-44f6-85d2-c3f9065272a6 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:59.632186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:59.645327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-3d58c6dd-637f-44f6-85d2-c3f9065272a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:59.645327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:37:59.645327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:37:59.645327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:37:59.645327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:37:59.645327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:37:59.645327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:37:59.645327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:37:59.645327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:37:59.645327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:37:59.645327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:37:59.645327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:37:59.645327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:37:59.645327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:37:59.645327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:37:59.645327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:37:59.646331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:37:59.646331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:37:59.646331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:37:59.646331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:37:59.646331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:37:59.646331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:37:59.646331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:37:59.646331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:37:59.646331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:37:59.646331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:37:59.646331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:37:59.646331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:37:59.646331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:37:59.646331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:37:59.646331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:37:59.646331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:37:59.646331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:37:59.647299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:37:59.647299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:37:59.647299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:37:59.647482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a593b51-c04d-49f3-812d-34f6cbe0fda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:59.647575 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 307/599] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:37:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:37:59.647644 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a593b51-c04d-49f3-812d-34f6cbe0fda6 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:59.647644 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a593b51-c04d-49f3-812d-34f6cbe0fda6 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:59.648235 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a593b51-c04d-49f3-812d-34f6cbe0fda6 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:59.648405 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97592c0c-0e83-4c08-88a0-37203bc45ddf None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:59.648559 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a593b51-c04d-49f3-812d-34f6cbe0fda6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:59.648650 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97592c0c-0e83-4c08-88a0-37203bc45ddf None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:59.653422 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a593b51-c04d-49f3-812d-34f6cbe0fda6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:59.653662 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97592c0c-0e83-4c08-88a0-37203bc45ddf None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:59.653841 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a593b51-c04d-49f3-812d-34f6cbe0fda6 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:59.653995 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 308/600] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:37:59.655071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97592c0c-0e83-4c08-88a0-37203bc45ddf None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:59.674514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 293/601] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:37:59.677019 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e05b3-1953-4edf-a31d-a73e00f15f1e None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:59.677303 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e05b3-1953-4edf-a31d-a73e00f15f1e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:59.677576 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e05b3-1953-4edf-a31d-a73e00f15f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:59.702681 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-e03e05b3-1953-4edf-a31d-a73e00f15f1e None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:37:59.703975 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e03e05b3-1953-4edf-a31d-a73e00f15f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:37:59.765277 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:37:59.794816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97592c0c-0e83-4c08-88a0-37203bc45ddf None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:37:59.795897 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 294/602] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:37:59.796579 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e05b3-1953-4edf-a31d-a73e00f15f1e None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:37:59.867178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e36f32-8498-4aa2-a092-9a7f847f582b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:37:59.869401 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e36f32-8498-4aa2-a092-9a7f847f582b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:37:59.870605 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e36f32-8498-4aa2-a092-9a7f847f582b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:37:59.872014 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e36f32-8498-4aa2-a092-9a7f847f582b None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:37:59.872900 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e36f32-8498-4aa2-a092-9a7f847f582b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:37:59.873763 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e36f32-8498-4aa2-a092-9a7f847f582b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:37:59.875481 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e36f32-8498-4aa2-a092-9a7f847f582b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:37:59.905854 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 309/603] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:37:59.907926 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e36f32-8498-4aa2-a092-9a7f847f582b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:00.046428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e05b3-1953-4edf-a31d-a73e00f15f1e None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:00.081186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:00.082510 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e36f32-8498-4aa2-a092-9a7f847f582b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:00.127702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a566c0-648d-418e-b94c-66366b807518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:00.130006 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a566c0-648d-418e-b94c-66366b807518 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:00.130285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a566c0-648d-418e-b94c-66366b807518 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:00.130554 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a566c0-648d-418e-b94c-66366b807518 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:00.135317 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d31498-1fb0-4164-ac85-66fe04d72737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:00.136251 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d31498-1fb0-4164-ac85-66fe04d72737 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:00.136752 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d31498-1fb0-4164-ac85-66fe04d72737 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:00.137219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d31498-1fb0-4164-ac85-66fe04d72737 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:00.138313 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a566c0-648d-418e-b94c-66366b807518 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:00.139912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a566c0-648d-418e-b94c-66366b807518 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:00.143797 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:00.147246 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d31498-1fb0-4164-ac85-66fe04d72737 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:00.149012 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d31498-1fb0-4164-ac85-66fe04d72737 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:00.156050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-c5a566c0-648d-418e-b94c-66366b807518 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:00.156050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:00.156050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:00.156050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:00.156050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:00.156050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:00.156050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:00.156050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:00.156050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:00.156050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:00.156050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:00.156050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:00.156050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:38:00.156050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:38:00.156050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:38:00.156983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:38:00.156983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:00.156983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:00.156983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:00.156983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:00.156983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:00.156983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:00.156983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:00.156983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:00.156983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:00.156983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:00.156983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:00.156983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:00.156983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:00.156983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:00.156983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:00.156983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:00.157873 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:38:00.157873 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:38:00.157873 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:38:00.157873 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:38:00.157873 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:38:00.157873 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:38:00.157873 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:00.157873 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:00.157873 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 310/604] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:37:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:00.158359 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a566c0-648d-418e-b94c-66366b807518 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:00.164958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-d5d31498-1fb0-4164-ac85-66fe04d72737 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:00.164958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:00.164958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:00.164958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:00.164958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:00.164958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:00.164958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:00.164958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:00.164958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:00.164958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:00.164958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:00.164958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:00.164958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:38:00.164958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:38:00.164958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:38:00.165879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:38:00.165879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:00.165879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:00.165879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:00.165879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:00.165879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:00.165879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:00.165879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:00.165879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:00.165879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:00.165879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:00.165879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:00.165879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:00.165879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:00.165879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:00.165879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:00.165879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:00.166752 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:38:00.166752 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:38:00.166752 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:38:00.166752 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:38:00.166752 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:38:00.166752 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:38:00.166752 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:00.166752 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:00.169140 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 295/605] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:37:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:00.170202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d31498-1fb0-4164-ac85-66fe04d72737 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:00.274646 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a566c0-648d-418e-b94c-66366b807518 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:00.277669 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d31498-1fb0-4164-ac85-66fe04d72737 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:00.313543 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223fd084-93ad-4201-8057-dc4c6b321e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:00.314187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fd084-93ad-4201-8057-dc4c6b321e91 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:00.314466 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fd084-93ad-4201-8057-dc4c6b321e91 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:00.314735 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fd084-93ad-4201-8057-dc4c6b321e91 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:00.315704 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bca503-e587-4be8-89c2-c1fe2090fafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:00.316525 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bca503-e587-4be8-89c2-c1fe2090fafa None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:00.317025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bca503-e587-4be8-89c2-c1fe2090fafa None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:00.317493 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bca503-e587-4be8-89c2-c1fe2090fafa None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:00.318409 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bca503-e587-4be8-89c2-c1fe2090fafa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:00.318929 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bca503-e587-4be8-89c2-c1fe2090fafa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:00.320590 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bca503-e587-4be8-89c2-c1fe2090fafa None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:00.326448 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223fd084-93ad-4201-8057-dc4c6b321e91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:38:00.333820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-223fd084-93ad-4201-8057-dc4c6b321e91 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:38:00.333820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:00.333820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:00.333820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:00.333820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:00.333820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:00.333820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:00.333820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:00.333820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:00.333820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:00.333820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:00.333820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:00.333820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:38:00.333820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:38:00.333820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:38:00.333820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:38:00.335111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:38:00.335111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:38:00.335111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:38:00.335111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:38:00.335111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:00.335111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:00.335111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:00.335111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:00.335111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:00.335111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:00.335111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:00.335111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:00.335111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:00.335111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:00.335111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:00.335111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:00.336216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:00.336216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:00.336216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:00.336216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:00.336216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:38:00.336216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:38:00.336216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:38:00.336216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:38:00.336216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:38:00.336216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:38:00.336216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:38:00.336216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:00.336216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 311/606] 10.222.0.71 () {56 vars in 1102 bytes} [Fri Aug 26 08:38:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:00.336216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223fd084-93ad-4201-8057-dc4c6b321e91 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:00.342488 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 296/607] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:38:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:00.343996 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bca503-e587-4be8-89c2-c1fe2090fafa None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:00.391079 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:00.416966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:00.464886 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223fd084-93ad-4201-8057-dc4c6b321e91 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:00.469489 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bca503-e587-4be8-89c2-c1fe2090fafa None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:00.506604 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c833568-c936-44ff-b30a-a5ca190834e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:00.507415 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c833568-c936-44ff-b30a-a5ca190834e2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:00.507794 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c833568-c936-44ff-b30a-a5ca190834e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:00.508162 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c833568-c936-44ff-b30a-a5ca190834e2 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:00.509095 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c833568-c936-44ff-b30a-a5ca190834e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:00.509582 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c833568-c936-44ff-b30a-a5ca190834e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:00.511161 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c833568-c936-44ff-b30a-a5ca190834e2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:00.511430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb967c-d63c-4e0f-8113-8209c330892b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:00.512480 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb967c-d63c-4e0f-8113-8209c330892b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:00.513216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb967c-d63c-4e0f-8113-8209c330892b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:00.513945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb967c-d63c-4e0f-8113-8209c330892b None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:00.515691 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb967c-d63c-4e0f-8113-8209c330892b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:00.516432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb967c-d63c-4e0f-8113-8209c330892b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:00.518825 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb967c-d63c-4e0f-8113-8209c330892b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:00.525747 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 297/608] 10.222.0.71 () {56 vars in 1103 bytes} [Fri Aug 26 08:38:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:00.526704 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c833568-c936-44ff-b30a-a5ca190834e2 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:00.538385 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 312/609] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:38:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:00.539707 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb967c-d63c-4e0f-8113-8209c330892b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:00.637085 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:00.648703 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c833568-c936-44ff-b30a-a5ca190834e2 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:00.662315 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb967c-d63c-4e0f-8113-8209c330892b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:00.688982 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:00.691169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04be09e-3a19-4b75-8bca-65b9e497c7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:00.692128 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04be09e-3a19-4b75-8bca-65b9e497c7d2 None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:00.692456 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04be09e-3a19-4b75-8bca-65b9e497c7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:00.692721 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04be09e-3a19-4b75-8bca-65b9e497c7d2 None admin] PATH_INFO: `/v3/projects/543d44293d894a7fbb83401f7a26dd9b/users/4aa8938a504c4c4f8815478e520482d8/roles/33ac296b51c143e7bb44211175752edb` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:00.707848 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d6816c-12d3-4a9b-836e-ada150a63baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:00.708882 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d6816c-12d3-4a9b-836e-ada150a63baa None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:00.709346 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d6816c-12d3-4a9b-836e-ada150a63baa None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:00.709793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d6816c-12d3-4a9b-836e-ada150a63baa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:00.711240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04be09e-3a19-4b75-8bca-65b9e497c7d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=543d44293d894a7fbb83401f7a26dd9b, user_id=4aa8938a504c4c4f8815478e520482d8, role_id=33ac296b51c143e7bb44211175752edb)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:00.712808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04be09e-3a19-4b75-8bca-65b9e497c7d2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:00.723727 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d6816c-12d3-4a9b-836e-ada150a63baa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:00.727270 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d6816c-12d3-4a9b-836e-ada150a63baa None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:00.735279 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 298/610] 10.222.0.71 () {58 vars in 1439 bytes} [Fri Aug 26 08:38:00 2022] PUT /identity/v3/projects/543d44293d894a7fbb83401f7a26dd9b/users/4aa8938a504c4c4f8815478e520482d8/roles/33ac296b51c143e7bb44211175752edb => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:38:00.736466 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04be09e-3a19-4b75-8bca-65b9e497c7d2 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:00.747316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-d6d6816c-12d3-4a9b-836e-ada150a63baa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:00.747316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:00.747316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:00.747316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:00.747316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:00.747316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:00.747316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:00.747316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:00.747316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:00.747316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:00.747316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:00.747316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:00.747316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:00.747316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:00.747316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:00.747316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:00.748341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:00.748341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:00.748341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:00.748341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:00.748341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:00.748341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:00.748341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:00.748341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:00.748341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:00.748341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:00.748341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:00.748341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:00.748341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:00.748341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:00.748341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:00.748341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:00.748341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:00.749256 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:00.749256 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:00.749256 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:00.751793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 313/611] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:00.757858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d6816c-12d3-4a9b-836e-ada150a63baa None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:00.888824 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d6816c-12d3-4a9b-836e-ada150a63baa None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:00.931060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4854f414-0a82-4d0c-8a83-b9c975399f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:00.931999 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854f414-0a82-4d0c-8a83-b9c975399f08 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:00.932523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854f414-0a82-4d0c-8a83-b9c975399f08 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:00.933076 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854f414-0a82-4d0c-8a83-b9c975399f08 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:00.934151 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4854f414-0a82-4d0c-8a83-b9c975399f08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:00.934648 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4854f414-0a82-4d0c-8a83-b9c975399f08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:00.936277 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4854f414-0a82-4d0c-8a83-b9c975399f08 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:00.952306 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 314/612] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:00.958502 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04be09e-3a19-4b75-8bca-65b9e497c7d2 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:00.959117 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4854f414-0a82-4d0c-8a83-b9c975399f08 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:00.998196 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:01.014733 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe5fea-b667-4d48-81a9-ce330ee93662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:01.015472 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe5fea-b667-4d48-81a9-ce330ee93662 None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:01.015995 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe5fea-b667-4d48-81a9-ce330ee93662 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:01.015995 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe5fea-b667-4d48-81a9-ce330ee93662 None admin] PATH_INFO: `/v3/projects/543d44293d894a7fbb83401f7a26dd9b/users/997b4ffc01fc467ebbd1092787f217f6/roles/33ac296b51c143e7bb44211175752edb` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:01.032375 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe5fea-b667-4d48-81a9-ce330ee93662 None admin] RBAC: Authorizing `identity:create_grant(project_id=543d44293d894a7fbb83401f7a26dd9b, user_id=997b4ffc01fc467ebbd1092787f217f6, role_id=33ac296b51c143e7bb44211175752edb)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:01.033855 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe5fea-b667-4d48-81a9-ce330ee93662 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:01.036409 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:01.053158 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 299/613] 10.222.0.71 () {58 vars in 1439 bytes} [Fri Aug 26 08:38:00 2022] PUT /identity/v3/projects/543d44293d894a7fbb83401f7a26dd9b/users/997b4ffc01fc467ebbd1092787f217f6/roles/33ac296b51c143e7bb44211175752edb => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:38:01.093268 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4854f414-0a82-4d0c-8a83-b9c975399f08 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:01.143735 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ab1cc-31ce-42fc-be03-768be5cf8de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:01.144403 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ab1cc-31ce-42fc-be03-768be5cf8de0 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:01.144676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ab1cc-31ce-42fc-be03-768be5cf8de0 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:01.144942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ab1cc-31ce-42fc-be03-768be5cf8de0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:01.153812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129ab1cc-31ce-42fc-be03-768be5cf8de0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:01.155608 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129ab1cc-31ce-42fc-be03-768be5cf8de0 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:01.174433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-129ab1cc-31ce-42fc-be03-768be5cf8de0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 08:38:01.174433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:01.174433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:01.174433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:01.174433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:01.174433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:01.174433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:01.174433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:01.174433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:01.174433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:01.174433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:01.174433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:01.174433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:38:01.174433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:38:01.174433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:38:01.174433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:38:01.175494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:01.175494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:01.175494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:38:01.175494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:01.175494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:38:01.175494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:01.175494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:01.175494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:01.175494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:01.175494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:01.175494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:01.175494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:01.175494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:01.175494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:01.175494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:01.175494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:01.175494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:01.175494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:01.176438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:01.176438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:01.176438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:38:01.176438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:38:01.176438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:38:01.176438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:38:01.176438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:38:01.176438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:38:01.176438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 08:38:01.176438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:01.176438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 315/614] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:38:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:01.182213 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe5fea-b667-4d48-81a9-ce330ee93662 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:01.313188 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe5fea-b667-4d48-81a9-ce330ee93662 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:01.350397 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a3978c-d5d5-4113-b549-46576f79130b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:01.351162 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3978c-d5d5-4113-b549-46576f79130b None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:01.351446 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3978c-d5d5-4113-b549-46576f79130b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:01.351768 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3978c-d5d5-4113-b549-46576f79130b None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:01.352596 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a3978c-d5d5-4113-b549-46576f79130b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:01.352908 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a3978c-d5d5-4113-b549-46576f79130b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:01.354359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a3978c-d5d5-4113-b549-46576f79130b None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:01.356645 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947c2db-b523-4a8e-884f-774f9bcdb2bc None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:01.356936 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947c2db-b523-4a8e-884f-774f9bcdb2bc None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:01.357259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f947c2db-b523-4a8e-884f-774f9bcdb2bc None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:01.359339 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 316/615] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:38:01.370646 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe6cf5-0074-406d-af21-bf401941c58b None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:01.370646 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe6cf5-0074-406d-af21-bf401941c58b None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:01.370646 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe6cf5-0074-406d-af21-bf401941c58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:01.380090 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 300/616] 10.222.0.71 () {56 vars in 1135 bytes} [Fri Aug 26 08:38:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:01.385529 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a3978c-d5d5-4113-b549-46576f79130b None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:01.415109 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-65fe6cf5-0074-406d-af21-bf401941c58b None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:01.416330 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-65fe6cf5-0074-406d-af21-bf401941c58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:01.447684 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:01.496599 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a3978c-d5d5-4113-b549-46576f79130b None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:01.541023 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6e8ba-6bc2-4523-b7b3-909617d3d6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:01.541666 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6e8ba-6bc2-4523-b7b3-909617d3d6ed None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:01.546103 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6e8ba-6bc2-4523-b7b3-909617d3d6ed None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:01.546103 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6e8ba-6bc2-4523-b7b3-909617d3d6ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:01.552289 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 317/617] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:38:01.556831 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6e8ba-6bc2-4523-b7b3-909617d3d6ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:01.558318 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6e8ba-6bc2-4523-b7b3-909617d3d6ed None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:01.577741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-60b6e8ba-6bc2-4523-b7b3-909617d3d6ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:01.577741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:01.577741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:01.577741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:01.577741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:01.577741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:01.577741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:01.577741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:01.577741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:01.577741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:01.577741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:01.577741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:01.577741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:01.577741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:01.577741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:01.577741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:01.578750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:01.578750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:01.578750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:01.578750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:01.578750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:01.578750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:01.578750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:01.578750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:01.578750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:01.578750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:01.578750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:01.578750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:01.578750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:01.578750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:01.578750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:01.578750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:01.578750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:01.579670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:01.579670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:01.579670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:01.579670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 301/618] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:38:01.585459 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe6cf5-0074-406d-af21-bf401941c58b None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:01.621469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:01.740682 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe6cf5-0074-406d-af21-bf401941c58b None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:01.782172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9243ad2-7b43-469d-a958-5f98fc429844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:01.782791 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9243ad2-7b43-469d-a958-5f98fc429844 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:01.783103 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9243ad2-7b43-469d-a958-5f98fc429844 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:01.783470 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9243ad2-7b43-469d-a958-5f98fc429844 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:01.784135 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9243ad2-7b43-469d-a958-5f98fc429844 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:01.784459 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9243ad2-7b43-469d-a958-5f98fc429844 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:01.785863 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9243ad2-7b43-469d-a958-5f98fc429844 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:01.798618 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 318/619] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:01.803938 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6e8ba-6bc2-4523-b7b3-909617d3d6ed None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:01.836392 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866b5cd-af89-4d60-a53b-f21717ee48f0 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:01.836802 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866b5cd-af89-4d60-a53b-f21717ee48f0 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:01.836964 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866b5cd-af89-4d60-a53b-f21717ee48f0 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:01.838570 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 319/620] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:01.844443 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ae1a6-eb52-4778-be52-a3dcbeda2fe2 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:01.844738 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ae1a6-eb52-4778-be52-a3dcbeda2fe2 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:01.845020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ae1a6-eb52-4778-be52-a3dcbeda2fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:01.864939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-510ae1a6-eb52-4778-be52-a3dcbeda2fe2 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:01.866189 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-510ae1a6-eb52-4778-be52-a3dcbeda2fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:01.943006 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6e8ba-6bc2-4523-b7b3-909617d3d6ed None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:01.943347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 320/621] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:38:01.944243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ae1a6-eb52-4778-be52-a3dcbeda2fe2 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:01.955281 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:01.959598 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:01.983715 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480a331-b995-42e6-a2dc-0f40eba72550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:01.984696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480a331-b995-42e6-a2dc-0f40eba72550 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:01.985156 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480a331-b995-42e6-a2dc-0f40eba72550 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:01.985634 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480a331-b995-42e6-a2dc-0f40eba72550 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:01.994481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f480a331-b995-42e6-a2dc-0f40eba72550 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:01.996596 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f480a331-b995-42e6-a2dc-0f40eba72550 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:02.010088 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-f480a331-b995-42e6-a2dc-0f40eba72550 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:02.010088 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:02.010088 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:02.010088 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:02.010088 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:02.010088 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:02.010088 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:02.010088 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:02.010088 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:02.010088 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:02.010088 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:02.010088 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:02.010088 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:38:02.010088 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:38:02.010088 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:38:02.011141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:38:02.011141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:02.011141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:02.011141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:02.011141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:02.011141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:02.011141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:02.011141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:02.011141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:02.011141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:02.011141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:02.011141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:02.011141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:02.011141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:02.011141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:02.011141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:02.011141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:02.012063 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:38:02.012063 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:38:02.012063 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:38:02.012063 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:38:02.012063 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:38:02.012063 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:38:02.012063 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:02.012063 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:02.014263 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 302/622] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:38:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:02.020585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480a331-b995-42e6-a2dc-0f40eba72550 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:02.063341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ae1a6-eb52-4778-be52-a3dcbeda2fe2 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:02.125155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad590b52-aab5-4765-b857-38b2125c6693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:02.125826 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad590b52-aab5-4765-b857-38b2125c6693 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:02.126132 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad590b52-aab5-4765-b857-38b2125c6693 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:02.126385 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad590b52-aab5-4765-b857-38b2125c6693 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:02.145917 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad590b52-aab5-4765-b857-38b2125c6693 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:38:02.150772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-ad590b52-aab5-4765-b857-38b2125c6693 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:38:02.150772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:02.150772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:02.150772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:02.150772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:02.150772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:02.150772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:02.150772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:02.150772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:02.150772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:02.150772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:02.150772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:02.150772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:38:02.150772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:38:02.150772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:38:02.150772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:38:02.151932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:38:02.151932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:38:02.151932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:38:02.151932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:38:02.151932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:02.151932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:02.151932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:02.151932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:02.151932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:02.151932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:02.151932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:02.151932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:02.151932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:02.151932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:02.151932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:02.151932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:02.152969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:02.152969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:02.152969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:02.152969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:02.152969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:38:02.152969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:38:02.152969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:38:02.152969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:38:02.152969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:38:02.152969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:38:02.152969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:38:02.152969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:02.152969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 321/623] 10.222.0.71 () {56 vars in 1102 bytes} [Fri Aug 26 08:38:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:02.160195 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad590b52-aab5-4765-b857-38b2125c6693 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:02.162729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480a331-b995-42e6-a2dc-0f40eba72550 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:02.203460 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020902a-4f56-48fb-9166-e5103da73e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:02.204488 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020902a-4f56-48fb-9166-e5103da73e97 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:02.205034 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020902a-4f56-48fb-9166-e5103da73e97 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:02.205504 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020902a-4f56-48fb-9166-e5103da73e97 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:02.206400 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020902a-4f56-48fb-9166-e5103da73e97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:02.206880 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020902a-4f56-48fb-9166-e5103da73e97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:02.208900 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020902a-4f56-48fb-9166-e5103da73e97 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:02.222547 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 303/624] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:38:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:02.228967 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020902a-4f56-48fb-9166-e5103da73e97 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:02.288271 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:02.341278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020902a-4f56-48fb-9166-e5103da73e97 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:02.344886 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad590b52-aab5-4765-b857-38b2125c6693 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:02.359378 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:02.383490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682a859-0dd3-4970-9bb4-9e51fa26772a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:02.383945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682a859-0dd3-4970-9bb4-9e51fa26772a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:02.384224 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682a859-0dd3-4970-9bb4-9e51fa26772a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:02.384498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682a859-0dd3-4970-9bb4-9e51fa26772a None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:02.385244 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a682a859-0dd3-4970-9bb4-9e51fa26772a None admin] RBAC: Adding query filter params (name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:02.385560 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a682a859-0dd3-4970-9bb4-9e51fa26772a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:02.385881 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24798aa-ca29-4200-be59-e6e8b6cab2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:02.386681 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24798aa-ca29-4200-be59-e6e8b6cab2b8 None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:02.387016 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a682a859-0dd3-4970-9bb4-9e51fa26772a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:02.387247 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24798aa-ca29-4200-be59-e6e8b6cab2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:02.387637 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24798aa-ca29-4200-be59-e6e8b6cab2b8 None admin] PATH_INFO: `/v3/projects/543d44293d894a7fbb83401f7a26dd9b/users/1723e961375b45b7b3bd99ddeb69683f/roles/33ac296b51c143e7bb44211175752edb` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:02.396669 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 322/625] 10.222.0.71 () {56 vars in 1103 bytes} [Fri Aug 26 08:38:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:02.398021 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682a859-0dd3-4970-9bb4-9e51fa26772a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:02.405901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24798aa-ca29-4200-be59-e6e8b6cab2b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=543d44293d894a7fbb83401f7a26dd9b, user_id=1723e961375b45b7b3bd99ddeb69683f, role_id=33ac296b51c143e7bb44211175752edb)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:02.407741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24798aa-ca29-4200-be59-e6e8b6cab2b8 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:02.424865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 304/626] 10.222.0.71 () {58 vars in 1439 bytes} [Fri Aug 26 08:38:02 2022] PUT /identity/v3/projects/543d44293d894a7fbb83401f7a26dd9b/users/1723e961375b45b7b3bd99ddeb69683f/roles/33ac296b51c143e7bb44211175752edb => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:38:02.426671 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24798aa-ca29-4200-be59-e6e8b6cab2b8 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:02.516076 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682a859-0dd3-4970-9bb4-9e51fa26772a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:02.551723 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:02.600085 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f04d9d-a9dc-4d6b-a11e-2b852d58e0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:02.601553 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f04d9d-a9dc-4d6b-a11e-2b852d58e0bf None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:02.603516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f04d9d-a9dc-4d6b-a11e-2b852d58e0bf None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:02.605255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f04d9d-a9dc-4d6b-a11e-2b852d58e0bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:02.612310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:02.626919 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24798aa-ca29-4200-be59-e6e8b6cab2b8 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:02.653937 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f04d9d-a9dc-4d6b-a11e-2b852d58e0bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:38:02.664321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-59f04d9d-a9dc-4d6b-a11e-2b852d58e0bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:38:02.664321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:02.664321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:02.664321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:02.664321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:02.664321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:02.664321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:02.664321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:02.664321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:02.664321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:02.664321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:02.664321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:02.664321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:38:02.664321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:38:02.664321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:38:02.664321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:38:02.666005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:38:02.666005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:38:02.666005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:38:02.666005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:38:02.666005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:02.666005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:02.666005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:02.666005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:02.666005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:02.666005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:02.666005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:02.666005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:02.666005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:02.666005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:02.666005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:02.666005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:02.667495 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:02.667495 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:02.667495 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:02.667495 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:02.667495 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:38:02.667495 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:38:02.667495 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:38:02.667495 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:38:02.667495 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:38:02.667495 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:38:02.667495 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:38:02.667495 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:02.673128 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 323/627] 10.222.0.71 () {56 vars in 1102 bytes} [Fri Aug 26 08:38:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:02.679192 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f04d9d-a9dc-4d6b-a11e-2b852d58e0bf None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:02.713385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae85223-cc34-4989-b38d-49f233df8443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:02.714097 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae85223-cc34-4989-b38d-49f233df8443 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:02.714397 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae85223-cc34-4989-b38d-49f233df8443 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:02.714691 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae85223-cc34-4989-b38d-49f233df8443 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:02.733300 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae85223-cc34-4989-b38d-49f233df8443 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:02.735343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae85223-cc34-4989-b38d-49f233df8443 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:02.764697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-aae85223-cc34-4989-b38d-49f233df8443 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:02.764697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:02.764697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:02.764697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:02.764697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:02.764697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:02.764697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:02.764697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:02.764697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:02.764697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:02.764697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:02.764697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:02.764697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:02.764697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:02.764697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:02.764697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:02.766460 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:02.766460 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:02.766460 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:02.766460 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:02.766460 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:02.766460 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:02.766460 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:02.766460 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:02.766460 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:02.766460 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:02.766460 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:02.766460 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:02.766460 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:02.766460 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:02.766460 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:02.766460 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:02.766460 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:02.768936 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:02.768936 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:02.768936 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:02.768936 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 305/628] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:02.771752 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae85223-cc34-4989-b38d-49f233df8443 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:02.887385 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f04d9d-a9dc-4d6b-a11e-2b852d58e0bf None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:02.942374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae85223-cc34-4989-b38d-49f233df8443 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:02.967154 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69418d52-cb56-4647-bc1f-f9a6e210c2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:02.967819 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69418d52-cb56-4647-bc1f-f9a6e210c2c5 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:02.968063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69418d52-cb56-4647-bc1f-f9a6e210c2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:02.968338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69418d52-cb56-4647-bc1f-f9a6e210c2c5 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:02.969030 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69418d52-cb56-4647-bc1f-f9a6e210c2c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:02.969336 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69418d52-cb56-4647-bc1f-f9a6e210c2c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:02.970732 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69418d52-cb56-4647-bc1f-f9a6e210c2c5 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:02.990584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 324/629] 10.222.0.71 () {56 vars in 1103 bytes} [Fri Aug 26 08:38:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:02.994707 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:02.995659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69418d52-cb56-4647-bc1f-f9a6e210c2c5 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:03.015637 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd2d82-69df-46ec-9e90-0dcf04f8a1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:03.016301 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd2d82-69df-46ec-9e90-0dcf04f8a1df None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:03.016578 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd2d82-69df-46ec-9e90-0dcf04f8a1df None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:03.016863 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd2d82-69df-46ec-9e90-0dcf04f8a1df None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:03.017563 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd2d82-69df-46ec-9e90-0dcf04f8a1df None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:03.017866 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd2d82-69df-46ec-9e90-0dcf04f8a1df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:03.019367 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd2d82-69df-46ec-9e90-0dcf04f8a1df None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:03.041215 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 306/630] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:03.048122 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd2d82-69df-46ec-9e90-0dcf04f8a1df None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:03.166616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd2d82-69df-46ec-9e90-0dcf04f8a1df None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:03.172552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:03.213595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eee509-e19b-4fca-9098-52f838895ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:03.214247 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eee509-e19b-4fca-9098-52f838895ada None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:03.214530 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eee509-e19b-4fca-9098-52f838895ada None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:03.214806 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eee509-e19b-4fca-9098-52f838895ada None admin] PATH_INFO: `/v3/users/nova` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:03.223604 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eee509-e19b-4fca-9098-52f838895ada None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:03.226120 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eee509-e19b-4fca-9098-52f838895ada None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:03.241670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-81eee509-e19b-4fca-9098-52f838895ada None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 08:38:03.241670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:03.241670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:03.241670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:03.241670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:03.241670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:03.241670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:03.241670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:03.241670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:03.241670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:03.241670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:03.241670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:03.241670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:38:03.241670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:38:03.241670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:38:03.241670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:38:03.243100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:03.243100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:03.243100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:38:03.243100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:03.243100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:38:03.243100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:03.243100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:03.243100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:03.243100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:03.243100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:03.243100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:03.243100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:03.243100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:03.243100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:03.243100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:03.243100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:03.243100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:03.243100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:03.244264 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:03.244264 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:03.244264 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:38:03.244264 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:38:03.244264 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:38:03.244264 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:38:03.244264 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:38:03.244264 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:38:03.244264 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 08:38:03.244264 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:03.244264 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 307/631] 10.222.0.71 () {56 vars in 1093 bytes} [Fri Aug 26 08:38:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:03.250299 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eee509-e19b-4fca-9098-52f838895ada None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:03.311577 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69418d52-cb56-4647-bc1f-f9a6e210c2c5 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:03.336312 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:03.351971 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3649cc0-8854-45b6-8ea1-57a262ca2724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:03.352621 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3649cc0-8854-45b6-8ea1-57a262ca2724 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:03.358190 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3649cc0-8854-45b6-8ea1-57a262ca2724 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:03.358553 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3649cc0-8854-45b6-8ea1-57a262ca2724 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:03.376952 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3649cc0-8854-45b6-8ea1-57a262ca2724 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:03.378559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3649cc0-8854-45b6-8ea1-57a262ca2724 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:03.394229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-d3649cc0-8854-45b6-8ea1-57a262ca2724 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:03.394229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:03.394229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:03.394229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:03.394229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:03.394229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:03.394229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:03.394229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:03.394229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:03.394229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:03.394229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:03.394229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:03.394229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:03.394229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:03.394229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:03.394229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:03.395354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:03.395354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:03.395354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:03.395354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:03.395354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:03.395354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:03.395354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:03.395354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:03.395354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:03.395354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:03.395354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:03.395354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:03.395354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:03.395354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:03.395354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:03.395354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:03.395354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:03.397066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:03.397066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:03.397066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:03.397066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 325/632] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:38:03.401551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3649cc0-8854-45b6-8ea1-57a262ca2724 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:03.458267 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eee509-e19b-4fca-9098-52f838895ada None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:03.498249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53917c91-a322-46e3-89d0-1cb7f318e2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:03.498874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53917c91-a322-46e3-89d0-1cb7f318e2e6 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:03.499282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53917c91-a322-46e3-89d0-1cb7f318e2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:03.499552 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53917c91-a322-46e3-89d0-1cb7f318e2e6 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:03.500316 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53917c91-a322-46e3-89d0-1cb7f318e2e6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:03.500622 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53917c91-a322-46e3-89d0-1cb7f318e2e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:03.502194 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53917c91-a322-46e3-89d0-1cb7f318e2e6 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:03.513978 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3649cc0-8854-45b6-8ea1-57a262ca2724 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:03.523967 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 308/633] 10.222.0.71 () {56 vars in 1133 bytes} [Fri Aug 26 08:38:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:03.524422 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:03.526436 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abea07-7322-4c05-be8f-18e6aedc7507 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:03.527116 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abea07-7322-4c05-be8f-18e6aedc7507 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:03.527596 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abea07-7322-4c05-be8f-18e6aedc7507 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:03.533541 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 309/634] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:03.534825 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93abea07-7322-4c05-be8f-18e6aedc7507 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:03.556396 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc5bec-5605-4654-8609-487cfae74ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:03.557078 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc5bec-5605-4654-8609-487cfae74ead None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:03.557355 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc5bec-5605-4654-8609-487cfae74ead None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:03.557666 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc5bec-5605-4654-8609-487cfae74ead None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:03.558326 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc5bec-5605-4654-8609-487cfae74ead None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:03.558624 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc5bec-5605-4654-8609-487cfae74ead None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:03.560163 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc5bec-5605-4654-8609-487cfae74ead None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:03.578121 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 326/635] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:03.579626 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2618075-92f9-43ce-a840-04cbfb650d35 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:03.579912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2618075-92f9-43ce-a840-04cbfb650d35 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:03.580183 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2618075-92f9-43ce-a840-04cbfb650d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:03.600624 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-b2618075-92f9-43ce-a840-04cbfb650d35 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:03.602194 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b2618075-92f9-43ce-a840-04cbfb650d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:03.674759 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93abea07-7322-4c05-be8f-18e6aedc7507 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:03.678491 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:03.691411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 327/636] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:38:03.692513 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2618075-92f9-43ce-a840-04cbfb650d35 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:03.740034 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877ab27-a92c-4fb4-99f4-5a1c3ef4e2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:03.741000 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877ab27-a92c-4fb4-99f4-5a1c3ef4e2c0 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:03.741538 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877ab27-a92c-4fb4-99f4-5a1c3ef4e2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:03.742013 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877ab27-a92c-4fb4-99f4-5a1c3ef4e2c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:03.819308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:03.824895 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2618075-92f9-43ce-a840-04cbfb650d35 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:03.840479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3877ab27-a92c-4fb4-99f4-5a1c3ef4e2c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:03.843014 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3877ab27-a92c-4fb4-99f4-5a1c3ef4e2c0 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:03.865237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1cad1-9bd5-4fea-a585-6e7877d6281d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:03.866392 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1cad1-9bd5-4fea-a585-6e7877d6281d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:03.866880 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1cad1-9bd5-4fea-a585-6e7877d6281d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:03.867401 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1cad1-9bd5-4fea-a585-6e7877d6281d None admin] PATH_INFO: `/v3/users/glance` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:03.875965 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1cad1-9bd5-4fea-a585-6e7877d6281d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:03.877883 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1cad1-9bd5-4fea-a585-6e7877d6281d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:03.892845 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-7da1cad1-9bd5-4fea-a585-6e7877d6281d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 26 08:38:03.892845 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:03.892845 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:03.892845 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:03.892845 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:03.892845 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:03.892845 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:03.892845 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:03.892845 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:03.892845 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:03.892845 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:03.892845 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:03.892845 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:38:03.892845 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:38:03.892845 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:38:03.892845 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:38:03.894153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:03.894153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:03.894153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:38:03.894153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:03.894153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:38:03.894153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:03.894153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:03.894153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:03.894153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:03.894153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:03.894153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:03.894153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:03.894153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:03.894153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:03.894153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:03.894153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:03.894153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:03.894153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:03.895168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:03.895168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:03.895168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:38:03.895168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:38:03.895168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:38:03.895168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:38:03.895168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:38:03.895168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:38:03.895168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 26 08:38:03.895168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:03.898205 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 328/637] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:38:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:03.900251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c3537-52ca-432b-85ee-d13bc9ae1fe9 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:03.900888 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c3537-52ca-432b-85ee-d13bc9ae1fe9 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:03.901351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c3537-52ca-432b-85ee-d13bc9ae1fe9 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:03.908315 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 329/638] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:03.909607 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6c3537-52ca-432b-85ee-d13bc9ae1fe9 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:03.974779 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 310/639] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:38:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:38:03.974779 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80856ffa-bbc3-4c06-8cc3-bb5658b558e9 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:03.974779 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80856ffa-bbc3-4c06-8cc3-bb5658b558e9 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:03.974779 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80856ffa-bbc3-4c06-8cc3-bb5658b558e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:03.983882 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:03.996910 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-80856ffa-bbc3-4c06-8cc3-bb5658b558e9 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:03.998026 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-80856ffa-bbc3-4c06-8cc3-bb5658b558e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:04.047798 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6c3537-52ca-432b-85ee-d13bc9ae1fe9 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:04.082498 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 311/640] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:38:04.083490 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80856ffa-bbc3-4c06-8cc3-bb5658b558e9 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:04.098333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358009a-83db-4f5a-9341-fe69f8e084fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:04.099936 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358009a-83db-4f5a-9341-fe69f8e084fd None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:04.100454 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358009a-83db-4f5a-9341-fe69f8e084fd None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:04.101441 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358009a-83db-4f5a-9341-fe69f8e084fd None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:04.103079 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358009a-83db-4f5a-9341-fe69f8e084fd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:04.103957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358009a-83db-4f5a-9341-fe69f8e084fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:04.106148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358009a-83db-4f5a-9341-fe69f8e084fd None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:04.132258 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 330/641] 10.222.0.71 () {56 vars in 1137 bytes} [Fri Aug 26 08:38:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:04.133587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358009a-83db-4f5a-9341-fe69f8e084fd None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:04.157218 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:04.227959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80856ffa-bbc3-4c06-8cc3-bb5658b558e9 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:04.271578 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc2d4e-29c2-4684-9190-8a02cf2cfb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:04.272258 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc2d4e-29c2-4684-9190-8a02cf2cfb4d None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:04.273206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc2d4e-29c2-4684-9190-8a02cf2cfb4d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:04.273782 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc2d4e-29c2-4684-9190-8a02cf2cfb4d None admin] PATH_INFO: `/v3/roles/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:04.277298 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:04.286880 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358009a-83db-4f5a-9341-fe69f8e084fd None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:04.289721 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc2d4e-29c2-4684-9190-8a02cf2cfb4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:38:04.295520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-80dc2d4e-29c2-4684-9190-8a02cf2cfb4d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:38:04.295520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:04.295520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:04.295520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:04.295520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:04.295520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:04.295520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:04.295520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:04.295520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:04.295520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:04.295520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:04.295520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:04.295520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:38:04.295520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:38:04.295520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:38:04.295520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:38:04.296482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:38:04.296482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:38:04.296482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:38:04.296482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:38:04.296482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:04.296482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:04.296482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:04.296482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:04.296482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:04.296482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:04.296482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:04.296482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:04.296482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:04.296482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:04.296482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:04.296482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:04.297337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:04.297337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:04.297337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:04.297337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:04.297337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:38:04.297337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:38:04.297337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:38:04.297337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:38:04.297337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:38:04.297337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:38:04.297337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:38:04.297337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:04.297337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 312/642] 10.222.0.71 () {56 vars in 1102 bytes} [Fri Aug 26 08:38:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:04.298041 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc2d4e-29c2-4684-9190-8a02cf2cfb4d None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:04.340154 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757e4c9-2975-4f32-9824-521a7ec1d076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:04.341067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757e4c9-2975-4f32-9824-521a7ec1d076 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:04.341825 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757e4c9-2975-4f32-9824-521a7ec1d076 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:04.342333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757e4c9-2975-4f32-9824-521a7ec1d076 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:04.363152 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2757e4c9-2975-4f32-9824-521a7ec1d076 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:04.364842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2757e4c9-2975-4f32-9824-521a7ec1d076 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:04.384537 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-2757e4c9-2975-4f32-9824-521a7ec1d076 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:04.384537 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:04.384537 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:04.384537 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:04.384537 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:04.384537 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:04.384537 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:04.384537 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:04.384537 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:04.384537 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:04.384537 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:04.384537 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:04.384537 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:04.384537 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:04.384537 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:04.384537 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:04.386676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:04.386676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:04.386676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:04.386676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:04.386676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:04.386676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:04.386676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:04.386676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:04.386676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:04.386676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:04.386676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:04.386676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:04.386676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:04.386676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:04.386676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:04.386676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:04.386676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:04.388299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:04.388299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:04.388299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:04.388771 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 331/643] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:04.389821 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757e4c9-2975-4f32-9824-521a7ec1d076 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:04.450089 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc2d4e-29c2-4684-9190-8a02cf2cfb4d None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:04.474261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:04.494381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570d115-4fab-40ff-a3bc-e9598950114b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:04.495200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570d115-4fab-40ff-a3bc-e9598950114b None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:04.495478 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570d115-4fab-40ff-a3bc-e9598950114b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:04.495746 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570d115-4fab-40ff-a3bc-e9598950114b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:04.519179 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757e4c9-2975-4f32-9824-521a7ec1d076 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:04.566316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49630c1c-c614-4b82-8f41-1242b4a7684f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:04.567259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49630c1c-c614-4b82-8f41-1242b4a7684f None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:04.567745 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49630c1c-c614-4b82-8f41-1242b4a7684f None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:04.568236 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49630c1c-c614-4b82-8f41-1242b4a7684f None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:04.569112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49630c1c-c614-4b82-8f41-1242b4a7684f None admin] RBAC: Adding query filter params (name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:04.569597 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49630c1c-c614-4b82-8f41-1242b4a7684f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:04.571691 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49630c1c-c614-4b82-8f41-1242b4a7684f None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:04.574563 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:04.589600 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 332/644] 10.222.0.71 () {56 vars in 1103 bytes} [Fri Aug 26 08:38:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:04.589600 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49630c1c-c614-4b82-8f41-1242b4a7684f None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:04.591819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570d115-4fab-40ff-a3bc-e9598950114b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:04.594278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570d115-4fab-40ff-a3bc-e9598950114b None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:04.732467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49630c1c-c614-4b82-8f41-1242b4a7684f None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:04.736079 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 313/645] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:38:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:38:04.736549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570d115-4fab-40ff-a3bc-e9598950114b None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:04.779850 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:04.787296 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce658af-67ca-403e-b01d-59928d201560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:04.788217 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce658af-67ca-403e-b01d-59928d201560 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:04.788498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce658af-67ca-403e-b01d-59928d201560 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:04.788779 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce658af-67ca-403e-b01d-59928d201560 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:04.790328 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce658af-67ca-403e-b01d-59928d201560 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:04.790328 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce658af-67ca-403e-b01d-59928d201560 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:04.791241 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce658af-67ca-403e-b01d-59928d201560 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:04.808960 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 333/646] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:04.808960 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce658af-67ca-403e-b01d-59928d201560 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:04.861245 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:04.869302 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570d115-4fab-40ff-a3bc-e9598950114b None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:04.920171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f6e6a-4511-4e42-b44b-2d0d277a4d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:04.920835 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f6e6a-4511-4e42-b44b-2d0d277a4d5f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:04.921113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f6e6a-4511-4e42-b44b-2d0d277a4d5f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:04.921378 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f6e6a-4511-4e42-b44b-2d0d277a4d5f None admin] PATH_INFO: `/v3/roles/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:04.933594 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f6e6a-4511-4e42-b44b-2d0d277a4d5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:38:04.935572 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce658af-67ca-403e-b01d-59928d201560 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:04.938432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-227f6e6a-4511-4e42-b44b-2d0d277a4d5f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:38:04.938432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:04.938432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:04.938432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:04.938432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:04.938432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:04.938432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:04.938432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:04.938432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:04.938432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:04.938432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:04.938432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:04.938432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:38:04.938432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:38:04.938432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:38:04.938432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:38:04.940308 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:38:04.940308 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:38:04.940308 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:38:04.940308 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:38:04.940308 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:04.940308 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:04.940308 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:04.940308 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:04.940308 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:04.940308 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:04.940308 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:04.940308 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:04.940308 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:04.940308 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:04.940308 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:04.940308 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:04.942278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:04.942278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:04.942278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:04.942278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:04.942278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:38:04.942278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:38:04.942278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:38:04.942278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:38:04.942278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:38:04.942278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:38:04.942278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:38:04.942278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:04.942278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 314/647] 10.222.0.71 () {56 vars in 1102 bytes} [Fri Aug 26 08:38:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:04.942278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f6e6a-4511-4e42-b44b-2d0d277a4d5f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:04.978820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e305332-45a0-4443-be3d-07b75f450cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:04.979503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e305332-45a0-4443-be3d-07b75f450cf9 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:04.979781 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e305332-45a0-4443-be3d-07b75f450cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:04.980058 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e305332-45a0-4443-be3d-07b75f450cf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:04.998576 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e305332-45a0-4443-be3d-07b75f450cf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:05.000194 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e305332-45a0-4443-be3d-07b75f450cf9 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:05.016489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-8e305332-45a0-4443-be3d-07b75f450cf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:05.016489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:05.016489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:05.016489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:05.016489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:05.016489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:05.016489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:05.016489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:05.016489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:05.016489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:05.016489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:05.016489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:05.016489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:05.016489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:05.016489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:05.016489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:05.018411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:05.018411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:05.018411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:05.018411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:05.018411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:05.018411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:05.018411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:05.018411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:05.018411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:05.018411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:05.018411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:05.018411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:05.018411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:05.018411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:05.018411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:05.018411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:05.018411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:05.020827 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:05.020827 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:05.020827 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:05.020827 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 334/648] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:05.020827 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e305332-45a0-4443-be3d-07b75f450cf9 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:05.059454 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f6e6a-4511-4e42-b44b-2d0d277a4d5f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:05.069013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:05.098605 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b5dbd-fed1-4f31-840e-87d1056e54b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:05.100062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b5dbd-fed1-4f31-840e-87d1056e54b8 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:05.100062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b5dbd-fed1-4f31-840e-87d1056e54b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:05.100514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b5dbd-fed1-4f31-840e-87d1056e54b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:05.112908 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075b5dbd-fed1-4f31-840e-87d1056e54b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:05.114358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075b5dbd-fed1-4f31-840e-87d1056e54b8 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:05.130253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-075b5dbd-fed1-4f31-840e-87d1056e54b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:05.130253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:05.130253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:05.130253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:05.130253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:05.130253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:05.130253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:05.130253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:05.130253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:05.130253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:05.130253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:05.130253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:05.130253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:05.130253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:05.130253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:05.130253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:05.136117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:05.136117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:05.136117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:05.136117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:05.136117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:05.136117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:05.136117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:05.136117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:05.136117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:05.136117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:05.136117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:05.136117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:05.136117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:05.136117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:05.136117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:05.136117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:05.136117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:05.137301 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:05.137301 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:05.137301 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:05.137301 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 315/649] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:05.137301 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c8985-c0ac-4b97-8eb9-deb3f632457a None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:05.137301 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c8985-c0ac-4b97-8eb9-deb3f632457a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:05.137301 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c8985-c0ac-4b97-8eb9-deb3f632457a None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:05.137301 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 316/650] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:05.137301 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c8985-c0ac-4b97-8eb9-deb3f632457a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:05.144127 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e305332-45a0-4443-be3d-07b75f450cf9 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:05.145323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:05.182449 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861df85-9e2b-4d5b-9b6a-ac3fe253c9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:05.183118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861df85-9e2b-4d5b-9b6a-ac3fe253c9ea None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:05.183396 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861df85-9e2b-4d5b-9b6a-ac3fe253c9ea None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:05.183665 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861df85-9e2b-4d5b-9b6a-ac3fe253c9ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:05.191565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861df85-9e2b-4d5b-9b6a-ac3fe253c9ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:05.193087 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861df85-9e2b-4d5b-9b6a-ac3fe253c9ea None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:05.204887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-8861df85-9e2b-4d5b-9b6a-ac3fe253c9ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:05.204887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:05.204887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:05.204887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:05.204887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:05.204887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:05.204887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:05.204887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:05.204887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:05.204887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:05.204887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:05.204887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:05.204887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:38:05.204887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:38:05.204887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:38:05.206026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:38:05.206026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:05.206026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:05.206026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:05.206026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:05.206026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:05.206026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:05.206026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:05.206026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:05.206026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:05.206026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:05.206026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:05.206026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:05.206026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:05.206026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:05.206026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:05.206026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:05.207097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:38:05.207097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:38:05.207097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:38:05.207097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:38:05.207097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:38:05.207097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:38:05.207097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:05.207097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:05.207097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 335/651] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:38:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:05.207097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861df85-9e2b-4d5b-9b6a-ac3fe253c9ea None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:05.259980 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c8985-c0ac-4b97-8eb9-deb3f632457a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:05.303925 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3538381-79ec-400b-85e9-3e7ce4a746b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:05.304540 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3538381-79ec-400b-85e9-3e7ce4a746b6 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:05.304812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3538381-79ec-400b-85e9-3e7ce4a746b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:05.305112 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3538381-79ec-400b-85e9-3e7ce4a746b6 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:05.305749 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3538381-79ec-400b-85e9-3e7ce4a746b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:05.306044 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3538381-79ec-400b-85e9-3e7ce4a746b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:05.307478 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3538381-79ec-400b-85e9-3e7ce4a746b6 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:05.316142 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 317/652] 10.222.0.71 () {56 vars in 1103 bytes} [Fri Aug 26 08:38:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:05.316975 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3538381-79ec-400b-85e9-3e7ce4a746b6 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:05.331325 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861df85-9e2b-4d5b-9b6a-ac3fe253c9ea None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:05.339726 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:05.377401 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5f360-b6ed-429b-9bcc-45ef762d6104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:05.378017 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5f360-b6ed-429b-9bcc-45ef762d6104 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:05.378293 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5f360-b6ed-429b-9bcc-45ef762d6104 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:05.378564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5f360-b6ed-429b-9bcc-45ef762d6104 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:05.379227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad5f360-b6ed-429b-9bcc-45ef762d6104 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:05.379527 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad5f360-b6ed-429b-9bcc-45ef762d6104 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:05.381033 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad5f360-b6ed-429b-9bcc-45ef762d6104 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:05.394005 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:05.397388 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 336/653] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:05.398848 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcd452-a3cb-4645-a57a-fa2d8af64665 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:05.399161 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcd452-a3cb-4645-a57a-fa2d8af64665 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:05.399431 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcd452-a3cb-4645-a57a-fa2d8af64665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:05.421294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e6bcd452-a3cb-4645-a57a-fa2d8af64665 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:05.422517 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e6bcd452-a3cb-4645-a57a-fa2d8af64665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:05.438123 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3538381-79ec-400b-85e9-3e7ce4a746b6 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:05.475376 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65121bd7-6b1f-4aab-a9b1-2cc685a84a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:05.475994 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65121bd7-6b1f-4aab-a9b1-2cc685a84a16 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:05.476274 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65121bd7-6b1f-4aab-a9b1-2cc685a84a16 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:05.476539 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65121bd7-6b1f-4aab-a9b1-2cc685a84a16 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:05.477201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65121bd7-6b1f-4aab-a9b1-2cc685a84a16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:05.477504 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65121bd7-6b1f-4aab-a9b1-2cc685a84a16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:05.478975 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65121bd7-6b1f-4aab-a9b1-2cc685a84a16 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:05.492016 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 318/654] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:05.493249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65121bd7-6b1f-4aab-a9b1-2cc685a84a16 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:05.495855 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 337/655] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:38:05.496504 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcd452-a3cb-4645-a57a-fa2d8af64665 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:05.627535 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:05.637406 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcd452-a3cb-4645-a57a-fa2d8af64665 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:05.644539 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65121bd7-6b1f-4aab-a9b1-2cc685a84a16 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:05.677064 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:05.680939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e6fb7-4f48-4f04-8bf6-9a59c50a9c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:05.681598 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e6fb7-4f48-4f04-8bf6-9a59c50a9c4f None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:05.681876 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e6fb7-4f48-4f04-8bf6-9a59c50a9c4f None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:05.682150 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e6fb7-4f48-4f04-8bf6-9a59c50a9c4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:05.686899 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246741ac-51fb-4737-89fd-12414fe3582a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:05.687555 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246741ac-51fb-4737-89fd-12414fe3582a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:05.687829 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246741ac-51fb-4737-89fd-12414fe3582a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:05.688094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246741ac-51fb-4737-89fd-12414fe3582a None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:05.688822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246741ac-51fb-4737-89fd-12414fe3582a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:05.689104 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246741ac-51fb-4737-89fd-12414fe3582a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:05.690663 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246741ac-51fb-4737-89fd-12414fe3582a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:05.695315 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e6fb7-4f48-4f04-8bf6-9a59c50a9c4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:05.697071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e6fb7-4f48-4f04-8bf6-9a59c50a9c4f None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:05.704223 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 319/656] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:38:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:05.705205 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246741ac-51fb-4737-89fd-12414fe3582a None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:05.719305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-200e6fb7-4f48-4f04-8bf6-9a59c50a9c4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:05.719305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:05.719305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:05.719305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:05.719305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:05.719305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:05.719305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:05.719305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:05.719305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:05.719305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:05.719305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:05.719305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:05.719305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:05.719305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:05.719305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:05.719305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:05.720341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:05.720341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:05.720341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:05.720341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:05.720341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:05.720341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:05.720341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:05.720341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:05.720341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:05.720341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:05.720341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:05.720341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:05.720341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:05.720341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:05.720341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:05.720341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:05.720341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:05.721297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:05.721297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:05.721297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:05.727632 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 338/657] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:05.727632 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e6fb7-4f48-4f04-8bf6-9a59c50a9c4f None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:05.832619 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246741ac-51fb-4737-89fd-12414fe3582a None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:05.848636 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e6fb7-4f48-4f04-8bf6-9a59c50a9c4f None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:05.874208 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784b689-48a5-4d63-b979-5ddac9fdbae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:05.875048 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784b689-48a5-4d63-b979-5ddac9fdbae1 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:05.875342 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784b689-48a5-4d63-b979-5ddac9fdbae1 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:05.875731 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784b689-48a5-4d63-b979-5ddac9fdbae1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:05.884987 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6784b689-48a5-4d63-b979-5ddac9fdbae1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:05.886457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6784b689-48a5-4d63-b979-5ddac9fdbae1 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:05.889689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699f59e-081a-4b2c-952f-c390228133b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:05.890345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699f59e-081a-4b2c-952f-c390228133b3 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:05.890624 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699f59e-081a-4b2c-952f-c390228133b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:05.890930 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7699f59e-081a-4b2c-952f-c390228133b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:05.899585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-6784b689-48a5-4d63-b979-5ddac9fdbae1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 08:38:05.899585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:05.899585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:05.899585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:05.899585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:05.899585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:05.899585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:05.899585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:05.899585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:05.899585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:05.899585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:05.899585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:05.899585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:38:05.899585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:38:05.899585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:38:05.899585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:38:05.901214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:05.901214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:05.901214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:38:05.901214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:05.901214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:38:05.901214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:05.901214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:05.901214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:05.901214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:05.901214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:05.901214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:05.901214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:05.901214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:05.901214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:05.901214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:05.901214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:05.901214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:05.901214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:05.902819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:05.902819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:05.902819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:38:05.902819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:38:05.902819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:38:05.902819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:38:05.902819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:38:05.902819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:38:05.902819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 08:38:05.902819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:05.902819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 320/658] 10.222.0.71 () {56 vars in 1102 bytes} [Fri Aug 26 08:38:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:05.902819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784b689-48a5-4d63-b979-5ddac9fdbae1 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:05.903581 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699f59e-081a-4b2c-952f-c390228133b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:05.904633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7699f59e-081a-4b2c-952f-c390228133b3 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:05.918334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-7699f59e-081a-4b2c-952f-c390228133b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:05.918334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:05.918334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:05.918334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:05.918334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:05.918334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:05.918334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:05.918334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:05.918334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:05.918334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:05.918334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:05.918334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:05.918334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:38:05.918334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:38:05.918334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:38:05.919523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:38:05.919523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:05.919523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:05.919523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:05.919523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:05.919523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:05.919523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:05.919523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:05.919523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:05.919523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:05.919523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:05.919523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:05.919523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:05.919523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:05.919523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:05.919523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:05.919523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:05.920450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:38:05.920450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:38:05.920450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:38:05.920450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:38:05.920450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:38:05.920450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:38:05.920450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:05.920450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:05.920450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 339/659] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:38:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:05.921328 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699f59e-081a-4b2c-952f-c390228133b3 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:05.925796 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:05.970192 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:06.034217 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784b689-48a5-4d63-b979-5ddac9fdbae1 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:06.048985 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7699f59e-081a-4b2c-952f-c390228133b3 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:06.073665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc63506-0868-468d-894d-068763f3e2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:06.074489 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc63506-0868-468d-894d-068763f3e2bb None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:06.074848 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc63506-0868-468d-894d-068763f3e2bb None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:06.075340 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc63506-0868-468d-894d-068763f3e2bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:06.076182 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc63506-0868-468d-894d-068763f3e2bb None admin] RBAC: Adding query filter params (role.id=33ac296b51c143e7bb44211175752edb, scope.project.id=543d44293d894a7fbb83401f7a26dd9b, user.id=f16d1768715a41648af3fe6afadd7477) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:06.076634 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc63506-0868-468d-894d-068763f3e2bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:06.078250 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc63506-0868-468d-894d-068763f3e2bb None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:06.085666 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 321/660] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:38:05 2022] GET /identity/v3/role_assignments?user.id=f16d1768715a41648af3fe6afadd7477&scope.project.id=543d44293d894a7fbb83401f7a26dd9b&role.id=33ac296b51c143e7bb44211175752edb => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:06.087065 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc63506-0868-468d-894d-068763f3e2bb None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:06.091844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b76a6-57f5-4910-be9a-75e90b416b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:06.092551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b76a6-57f5-4910-be9a-75e90b416b86 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:06.092838 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b76a6-57f5-4910-be9a-75e90b416b86 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:06.093116 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b76a6-57f5-4910-be9a-75e90b416b86 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:06.093946 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b76a6-57f5-4910-be9a-75e90b416b86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:06.094297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b76a6-57f5-4910-be9a-75e90b416b86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:06.095784 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b76a6-57f5-4910-be9a-75e90b416b86 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:06.119205 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 340/661] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:06.119205 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b76a6-57f5-4910-be9a-75e90b416b86 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:06.228829 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc63506-0868-468d-894d-068763f3e2bb None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:06.229979 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:06.248065 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:06.279994 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f5fc1-2d9b-45ee-9b78-ae125e2dc19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:06.280727 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f5fc1-2d9b-45ee-9b78-ae125e2dc19c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:06.281287 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f5fc1-2d9b-45ee-9b78-ae125e2dc19c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:06.281586 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f5fc1-2d9b-45ee-9b78-ae125e2dc19c None admin] PATH_INFO: `/v3/roles/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:06.294624 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b76a6-57f5-4910-be9a-75e90b416b86 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:06.298093 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f5fc1-2d9b-45ee-9b78-ae125e2dc19c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:38:06.303186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-ad5f5fc1-2d9b-45ee-9b78-ae125e2dc19c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:38:06.303186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:06.303186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:06.303186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:06.303186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:06.303186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:06.303186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:06.303186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:06.303186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:06.303186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:06.303186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:06.303186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:06.303186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:38:06.303186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:38:06.303186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:38:06.303186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:38:06.304451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:38:06.304451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:38:06.304451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:38:06.304451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:38:06.304451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:06.304451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:06.304451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:06.304451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:06.304451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:06.304451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:06.304451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:06.304451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:06.304451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:06.304451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:06.304451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:06.304451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:06.306083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:06.306083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:06.306083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:06.306083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:06.306083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:38:06.306083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:38:06.306083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:38:06.306083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:38:06.306083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:38:06.306083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:38:06.306083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:38:06.306083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:06.306083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 322/662] 10.222.0.71 () {56 vars in 1102 bytes} [Fri Aug 26 08:38:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:06.306083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f5fc1-2d9b-45ee-9b78-ae125e2dc19c None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:06.366960 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4dfa53-60bf-463e-b954-57de8620aafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:06.368334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dfa53-60bf-463e-b954-57de8620aafc None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:06.368668 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dfa53-60bf-463e-b954-57de8620aafc None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:06.368983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dfa53-60bf-463e-b954-57de8620aafc None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:06.370278 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4dfa53-60bf-463e-b954-57de8620aafc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:06.370621 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4dfa53-60bf-463e-b954-57de8620aafc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:06.373004 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4dfa53-60bf-463e-b954-57de8620aafc None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:06.399895 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 341/663] 10.222.0.71 () {56 vars in 1139 bytes} [Fri Aug 26 08:38:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:06.400769 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4dfa53-60bf-463e-b954-57de8620aafc None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:06.467559 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f5fc1-2d9b-45ee-9b78-ae125e2dc19c None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:06.521507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cda2b8-fe20-4143-bee7-055fc9252425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:06.522532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cda2b8-fe20-4143-bee7-055fc9252425 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:06.522846 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cda2b8-fe20-4143-bee7-055fc9252425 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:06.523111 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cda2b8-fe20-4143-bee7-055fc9252425 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:06.524273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cda2b8-fe20-4143-bee7-055fc9252425 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:06.524595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cda2b8-fe20-4143-bee7-055fc9252425 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:06.526104 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cda2b8-fe20-4143-bee7-055fc9252425 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:06.546119 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 323/664] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:38:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:06.546119 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cda2b8-fe20-4143-bee7-055fc9252425 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:06.560947 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4dfa53-60bf-463e-b954-57de8620aafc None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:06.585849 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:06.591451 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:06.610070 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af778ba6-8b3d-4e94-8635-daa9bd7d8cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:06.610070 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af778ba6-8b3d-4e94-8635-daa9bd7d8cc0 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:06.610070 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af778ba6-8b3d-4e94-8635-daa9bd7d8cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:06.610070 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af778ba6-8b3d-4e94-8635-daa9bd7d8cc0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:06.619756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af778ba6-8b3d-4e94-8635-daa9bd7d8cc0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:06.621454 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af778ba6-8b3d-4e94-8635-daa9bd7d8cc0 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:06.642084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-af778ba6-8b3d-4e94-8635-daa9bd7d8cc0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 08:38:06.642084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:06.642084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:06.642084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:06.642084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:06.642084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:06.642084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:06.642084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:06.642084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:06.642084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:06.642084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:06.642084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:06.642084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:38:06.642084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:38:06.642084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:38:06.642084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:38:06.643146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:06.643146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:06.643146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:38:06.643146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:06.643146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:38:06.643146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:06.643146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:06.643146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:06.643146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:06.643146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:06.643146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:06.643146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:06.643146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:06.643146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:06.643146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:06.643146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:06.643146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:06.643146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:06.644075 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:06.644075 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:06.644075 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:38:06.644075 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:38:06.644075 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:38:06.644075 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:38:06.644075 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:38:06.644075 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:38:06.644075 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 08:38:06.644075 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:06.644075 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 342/665] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:38:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:06.645384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af778ba6-8b3d-4e94-8635-daa9bd7d8cc0 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:06.701365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cda2b8-fe20-4143-bee7-055fc9252425 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:06.758000 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feac643-8d5c-4cc9-9da2-b943bdc4a683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:06.758629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feac643-8d5c-4cc9-9da2-b943bdc4a683 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:06.758945 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feac643-8d5c-4cc9-9da2-b943bdc4a683 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:06.759361 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feac643-8d5c-4cc9-9da2-b943bdc4a683 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:06.759969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feac643-8d5c-4cc9-9da2-b943bdc4a683 None admin] RBAC: Adding query filter params (name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:06.760613 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feac643-8d5c-4cc9-9da2-b943bdc4a683 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:06.761678 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feac643-8d5c-4cc9-9da2-b943bdc4a683 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:06.776162 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 324/666] 10.222.0.71 () {56 vars in 1103 bytes} [Fri Aug 26 08:38:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:06.776162 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feac643-8d5c-4cc9-9da2-b943bdc4a683 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:06.783677 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af778ba6-8b3d-4e94-8635-daa9bd7d8cc0 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:06.823921 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2eb30-e899-4024-8921-df02d3231902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:06.824534 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2eb30-e899-4024-8921-df02d3231902 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:06.824805 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2eb30-e899-4024-8921-df02d3231902 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:06.825075 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2eb30-e899-4024-8921-df02d3231902 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:06.888475 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:06.912710 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2eb30-e899-4024-8921-df02d3231902 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:06.914063 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:06.914181 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2eb30-e899-4024-8921-df02d3231902 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:06.925289 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feac643-8d5c-4cc9-9da2-b943bdc4a683 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:06.978561 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f63081-7f54-4631-a8ae-8d7acc85b9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:06.979204 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f63081-7f54-4631-a8ae-8d7acc85b9af None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:06.979477 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f63081-7f54-4631-a8ae-8d7acc85b9af None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:06.979743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f63081-7f54-4631-a8ae-8d7acc85b9af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:06.980483 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f63081-7f54-4631-a8ae-8d7acc85b9af None admin] RBAC: Adding query filter params (role.id=33ac296b51c143e7bb44211175752edb, scope.project.id=543d44293d894a7fbb83401f7a26dd9b, user.id=0d178c717a3e49319545e70cb58cbbed) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:06.980809 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f63081-7f54-4631-a8ae-8d7acc85b9af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:06.982348 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f63081-7f54-4631-a8ae-8d7acc85b9af None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:06.999109 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 325/667] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:38:06 2022] GET /identity/v3/role_assignments?user.id=0d178c717a3e49319545e70cb58cbbed&scope.project.id=543d44293d894a7fbb83401f7a26dd9b&role.id=33ac296b51c143e7bb44211175752edb => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:06.999109 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f63081-7f54-4631-a8ae-8d7acc85b9af None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:07.097629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 343/668] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:38:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:38:07.098509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2eb30-e899-4024-8921-df02d3231902 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:07.164343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f63081-7f54-4631-a8ae-8d7acc85b9af None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:07.210054 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9bb292-ccf9-4049-8847-6fd8d48e06dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:07.210718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9bb292-ccf9-4049-8847-6fd8d48e06dc None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:07.211032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9bb292-ccf9-4049-8847-6fd8d48e06dc None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:07.211304 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9bb292-ccf9-4049-8847-6fd8d48e06dc None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:07.212620 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9bb292-ccf9-4049-8847-6fd8d48e06dc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:07.213678 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9bb292-ccf9-4049-8847-6fd8d48e06dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:07.215166 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9bb292-ccf9-4049-8847-6fd8d48e06dc None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:07.220278 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:07.241492 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 326/669] 10.222.0.71 () {56 vars in 1137 bytes} [Fri Aug 26 08:38:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:07.246914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9bb292-ccf9-4049-8847-6fd8d48e06dc None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:07.258175 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2eb30-e899-4024-8921-df02d3231902 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:07.276724 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:07.313376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86dcbb1-3576-410d-900e-3ef537266a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:07.314100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86dcbb1-3576-410d-900e-3ef537266a86 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:07.314382 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86dcbb1-3576-410d-900e-3ef537266a86 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:07.314653 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86dcbb1-3576-410d-900e-3ef537266a86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:07.328829 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86dcbb1-3576-410d-900e-3ef537266a86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:07.330321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86dcbb1-3576-410d-900e-3ef537266a86 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:07.350713 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-e86dcbb1-3576-410d-900e-3ef537266a86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:07.350713 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:07.350713 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:07.350713 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:07.350713 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:07.350713 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:07.350713 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:07.350713 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:07.350713 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:07.350713 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:07.350713 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:07.350713 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:07.350713 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:07.350713 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:07.350713 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:07.350713 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:07.351774 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:07.351774 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:07.351774 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:07.351774 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:07.351774 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:07.351774 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:07.351774 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:07.351774 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:07.351774 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:07.351774 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:07.351774 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:07.351774 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:07.351774 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:07.351774 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:07.351774 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:07.351774 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:07.351774 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:07.352664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:07.352664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:07.352664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:07.354918 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 344/670] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:07.356087 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86dcbb1-3576-410d-900e-3ef537266a86 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:07.397519 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9bb292-ccf9-4049-8847-6fd8d48e06dc None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:07.444748 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a45747-6e59-480f-90c5-f4fd6251f14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:07.445392 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a45747-6e59-480f-90c5-f4fd6251f14f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:07.445665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a45747-6e59-480f-90c5-f4fd6251f14f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:07.445936 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a45747-6e59-480f-90c5-f4fd6251f14f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:07.460765 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a45747-6e59-480f-90c5-f4fd6251f14f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:07.462928 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a45747-6e59-480f-90c5-f4fd6251f14f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:07.484315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-b2a45747-6e59-480f-90c5-f4fd6251f14f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:07.484315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:07.484315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:07.484315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:07.484315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:07.484315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:07.484315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:07.484315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:07.484315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:07.484315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:07.484315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:07.484315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:07.484315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:07.484315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:07.484315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:07.484315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:07.485327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:07.485327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:07.485327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:07.485327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:07.485327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:07.485327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:07.485327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:07.485327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:07.485327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:07.485327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:07.485327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:07.485327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:07.485327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:07.485327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:07.485327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:07.485327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:07.485327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:07.486225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:07.486225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:07.486225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:07.486225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 327/671] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:07.486913 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a45747-6e59-480f-90c5-f4fd6251f14f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:07.488583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86dcbb1-3576-410d-900e-3ef537266a86 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:07.535992 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774fbeb-c17e-4ae7-94b5-03f3dd78a148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:07.537149 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774fbeb-c17e-4ae7-94b5-03f3dd78a148 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:07.537714 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774fbeb-c17e-4ae7-94b5-03f3dd78a148 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:07.538306 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774fbeb-c17e-4ae7-94b5-03f3dd78a148 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:07.541955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:07.596866 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:07.636489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774fbeb-c17e-4ae7-94b5-03f3dd78a148 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:07.638603 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774fbeb-c17e-4ae7-94b5-03f3dd78a148 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:07.645753 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a45747-6e59-480f-90c5-f4fd6251f14f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:07.697040 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29553f64-c026-4efd-8d5b-7f0b0498feb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:07.698029 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29553f64-c026-4efd-8d5b-7f0b0498feb3 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:07.698507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29553f64-c026-4efd-8d5b-7f0b0498feb3 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:07.699038 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29553f64-c026-4efd-8d5b-7f0b0498feb3 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:07.699905 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29553f64-c026-4efd-8d5b-7f0b0498feb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:07.700436 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29553f64-c026-4efd-8d5b-7f0b0498feb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:07.702058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29553f64-c026-4efd-8d5b-7f0b0498feb3 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:07.717302 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 328/672] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:07.718540 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29553f64-c026-4efd-8d5b-7f0b0498feb3 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:07.774567 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 345/673] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:38:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:38:07.775493 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774fbeb-c17e-4ae7-94b5-03f3dd78a148 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:07.819873 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:07.855116 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29553f64-c026-4efd-8d5b-7f0b0498feb3 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:07.899487 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:07.908256 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774fbeb-c17e-4ae7-94b5-03f3dd78a148 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:07.913864 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da566aa7-7065-4cde-923a-106ce641f679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:07.914811 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da566aa7-7065-4cde-923a-106ce641f679 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:07.915212 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da566aa7-7065-4cde-923a-106ce641f679 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:07.915641 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da566aa7-7065-4cde-923a-106ce641f679 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:07.916820 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da566aa7-7065-4cde-923a-106ce641f679 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:07.917270 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da566aa7-7065-4cde-923a-106ce641f679 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:07.919889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da566aa7-7065-4cde-923a-106ce641f679 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:07.940498 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 329/674] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:07.942035 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da566aa7-7065-4cde-923a-106ce641f679 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:07.952872 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7aa68-ace8-4d8e-930d-35786c528656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:07.953872 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7aa68-ace8-4d8e-930d-35786c528656 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:07.954376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7aa68-ace8-4d8e-930d-35786c528656 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:07.954857 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a7aa68-ace8-4d8e-930d-35786c528656 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:07.964299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a7aa68-ace8-4d8e-930d-35786c528656 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:07.966345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a7aa68-ace8-4d8e-930d-35786c528656 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:07.984659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-70a7aa68-ace8-4d8e-930d-35786c528656 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 08:38:07.984659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:07.984659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:07.984659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:07.984659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:07.984659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:07.984659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:07.984659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:07.984659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:07.984659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:07.984659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:07.984659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:07.984659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:38:07.984659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:38:07.984659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:38:07.984659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:38:07.985896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:07.985896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:07.985896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:38:07.985896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:07.985896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:38:07.985896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:07.985896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:07.985896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:07.985896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:07.985896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:07.985896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:07.985896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:07.985896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:07.985896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:07.985896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:07.985896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:07.985896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:07.985896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:07.987084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:07.987084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:07.987084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:38:07.987084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:38:07.987084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:38:07.987084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:38:07.987084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:38:07.987084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:38:07.987084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 08:38:07.987084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:07.994852 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 346/675] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:38:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:07.994852 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7aa68-ace8-4d8e-930d-35786c528656 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:08.101249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da566aa7-7065-4cde-923a-106ce641f679 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:08.139712 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:08.150049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd61685-aad7-45f7-9830-6c034713ac68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:08.150775 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd61685-aad7-45f7-9830-6c034713ac68 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:08.151174 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd61685-aad7-45f7-9830-6c034713ac68 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:08.151535 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd61685-aad7-45f7-9830-6c034713ac68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:08.152479 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a7aa68-ace8-4d8e-930d-35786c528656 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:08.167638 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd61685-aad7-45f7-9830-6c034713ac68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:08.169590 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd61685-aad7-45f7-9830-6c034713ac68 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:08.192248 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-4dd61685-aad7-45f7-9830-6c034713ac68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:08.192248 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:08.192248 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:08.192248 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:08.192248 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:08.192248 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:08.192248 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:08.192248 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:08.192248 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:08.192248 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:08.192248 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:08.192248 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:08.192248 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:08.192248 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:08.192248 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:08.192248 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:08.193359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:08.193359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:08.193359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:08.193359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:08.193359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:08.193359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:08.193359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:08.193359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:08.193359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:08.193359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:08.193359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:08.193359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:08.193359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:08.193359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:08.193359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:08.193359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:08.193359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:08.194249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:08.194249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:08.194249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:08.194249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 330/676] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:08.195322 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd61685-aad7-45f7-9830-6c034713ac68 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:08.199037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe8939-9e6a-4bac-ae84-379058241e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:08.200981 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe8939-9e6a-4bac-ae84-379058241e57 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:08.201480 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe8939-9e6a-4bac-ae84-379058241e57 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:08.201979 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe8939-9e6a-4bac-ae84-379058241e57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:08.212864 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe8939-9e6a-4bac-ae84-379058241e57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:08.216370 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe8939-9e6a-4bac-ae84-379058241e57 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:08.241030 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-d3fe8939-9e6a-4bac-ae84-379058241e57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:08.241030 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:08.241030 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:08.241030 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:08.241030 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:08.241030 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:08.241030 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:08.241030 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:08.241030 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:08.241030 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:08.241030 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:08.241030 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:08.241030 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:38:08.241030 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:38:08.241030 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:38:08.242588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:38:08.242588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:08.242588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:08.242588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:08.242588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:08.242588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:08.242588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:08.242588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:08.242588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:08.242588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:08.242588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:08.242588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:08.242588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:08.242588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:08.242588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:08.242588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:08.242588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:08.244874 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:38:08.244874 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:38:08.244874 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:38:08.244874 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:38:08.244874 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:38:08.244874 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:38:08.244874 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:08.244874 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:08.254588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 347/677] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:38:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:08.254588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe8939-9e6a-4bac-ae84-379058241e57 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:08.259815 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:08.354337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd61685-aad7-45f7-9830-6c034713ac68 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:08.418084 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc05bb8-3227-4e7d-bb70-c534021a92a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:08.418733 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc05bb8-3227-4e7d-bb70-c534021a92a7 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:08.418983 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc05bb8-3227-4e7d-bb70-c534021a92a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:08.419192 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc05bb8-3227-4e7d-bb70-c534021a92a7 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:08.419959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc05bb8-3227-4e7d-bb70-c534021a92a7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:08.420736 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc05bb8-3227-4e7d-bb70-c534021a92a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:08.422675 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc05bb8-3227-4e7d-bb70-c534021a92a7 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:08.441553 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe8939-9e6a-4bac-ae84-379058241e57 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:08.452764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 331/678] 10.222.0.71 () {56 vars in 1135 bytes} [Fri Aug 26 08:38:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:08.452764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc05bb8-3227-4e7d-bb70-c534021a92a7 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:08.485512 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7bf5f5-8db4-428d-bd35-b197b307a8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:08.486249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7bf5f5-8db4-428d-bd35-b197b307a8bb None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:08.486679 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7bf5f5-8db4-428d-bd35-b197b307a8bb None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:08.486989 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7bf5f5-8db4-428d-bd35-b197b307a8bb None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:08.487673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7bf5f5-8db4-428d-bd35-b197b307a8bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:08.488086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7bf5f5-8db4-428d-bd35-b197b307a8bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:08.489940 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7bf5f5-8db4-428d-bd35-b197b307a8bb None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:08.507494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 348/679] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:08.507494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7bf5f5-8db4-428d-bd35-b197b307a8bb None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:08.524714 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:08.587514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:08.591071 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc05bb8-3227-4e7d-bb70-c534021a92a7 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:08.634759 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cf84f-8928-4b5d-ae73-c960a8567820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:08.635819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cf84f-8928-4b5d-ae73-c960a8567820 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:08.636266 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cf84f-8928-4b5d-ae73-c960a8567820 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:08.636781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cf84f-8928-4b5d-ae73-c960a8567820 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:08.638037 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60cf84f-8928-4b5d-ae73-c960a8567820 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:08.639232 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60cf84f-8928-4b5d-ae73-c960a8567820 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:08.640651 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60cf84f-8928-4b5d-ae73-c960a8567820 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:08.659168 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 332/680] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:38:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:08.659168 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c8ab3-c34e-4c46-99da-ad9102cd8190 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:08.659168 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c8ab3-c34e-4c46-99da-ad9102cd8190 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:08.659168 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c8ab3-c34e-4c46-99da-ad9102cd8190 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:08.659168 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 333/681] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:08.659168 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c8ab3-c34e-4c46-99da-ad9102cd8190 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:08.671162 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7bf5f5-8db4-428d-bd35-b197b307a8bb None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:08.718437 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c289d64a-1894-4690-8a13-c0e5688650c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:08.719175 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289d64a-1894-4690-8a13-c0e5688650c5 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:08.719455 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289d64a-1894-4690-8a13-c0e5688650c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:08.719725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c289d64a-1894-4690-8a13-c0e5688650c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:08.809427 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c289d64a-1894-4690-8a13-c0e5688650c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:08.811302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c289d64a-1894-4690-8a13-c0e5688650c5 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:08.821777 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c8ab3-c34e-4c46-99da-ad9102cd8190 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:08.831031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:08.869236 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35dbe5a-8321-4a17-b36f-d47739a2f7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:08.869986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35dbe5a-8321-4a17-b36f-d47739a2f7fc None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:08.870276 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35dbe5a-8321-4a17-b36f-d47739a2f7fc None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:08.870580 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35dbe5a-8321-4a17-b36f-d47739a2f7fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:08.878391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35dbe5a-8321-4a17-b36f-d47739a2f7fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:08.880142 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35dbe5a-8321-4a17-b36f-d47739a2f7fc None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:08.896015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-c35dbe5a-8321-4a17-b36f-d47739a2f7fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:08.896015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:08.896015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:08.896015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:08.896015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:08.896015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:08.896015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:08.896015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:08.896015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:08.896015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:08.896015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:08.896015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:08.896015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:38:08.896015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:38:08.896015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:38:08.896979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:38:08.896979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:08.896979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:08.896979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:08.896979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:08.896979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:08.896979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:08.896979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:08.896979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:08.896979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:08.896979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:08.896979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:08.896979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:08.896979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:08.896979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:08.896979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:08.896979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:08.897889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:38:08.897889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:38:08.897889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:38:08.897889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:38:08.897889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:38:08.897889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:38:08.897889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:08.897889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:08.897889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 334/682] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:38:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:08.898384 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35dbe5a-8321-4a17-b36f-d47739a2f7fc None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:08.928935 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:08.959893 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 349/683] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:38:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 08:38:08.959893 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9314f-36ee-42da-85e1-ae33ca9f4b2f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:08.959893 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9314f-36ee-42da-85e1-ae33ca9f4b2f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:08.959893 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9314f-36ee-42da-85e1-ae33ca9f4b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:08.977174 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-1df9314f-36ee-42da-85e1-ae33ca9f4b2f None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:08.978442 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1df9314f-36ee-42da-85e1-ae33ca9f4b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:09.060459 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35dbe5a-8321-4a17-b36f-d47739a2f7fc None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:09.073049 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 350/684] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:38:09.073928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9314f-36ee-42da-85e1-ae33ca9f4b2f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:09.112644 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d793b-d5a6-408a-827f-d2039b13cd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:09.113274 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d793b-d5a6-408a-827f-d2039b13cd74 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:09.113556 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d793b-d5a6-408a-827f-d2039b13cd74 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:09.113831 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d793b-d5a6-408a-827f-d2039b13cd74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:09.114615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d793b-d5a6-408a-827f-d2039b13cd74 None admin] RBAC: Adding query filter params (role.id=33ac296b51c143e7bb44211175752edb, scope.project.id=543d44293d894a7fbb83401f7a26dd9b, user.id=4aa8938a504c4c4f8815478e520482d8) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:09.114941 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d793b-d5a6-408a-827f-d2039b13cd74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:09.116524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d793b-d5a6-408a-827f-d2039b13cd74 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:09.131129 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 335/685] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:38:08 2022] GET /identity/v3/role_assignments?user.id=4aa8938a504c4c4f8815478e520482d8&scope.project.id=543d44293d894a7fbb83401f7a26dd9b&role.id=33ac296b51c143e7bb44211175752edb => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:09.131129 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d793b-d5a6-408a-827f-d2039b13cd74 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:09.141135 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:09.244682 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9314f-36ee-42da-85e1-ae33ca9f4b2f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:09.308808 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984fe911-653e-480c-99b4-307284c06b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:09.309449 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fe911-653e-480c-99b4-307284c06b87 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:09.309726 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fe911-653e-480c-99b4-307284c06b87 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:09.309995 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fe911-653e-480c-99b4-307284c06b87 None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:09.310750 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984fe911-653e-480c-99b4-307284c06b87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:09.311083 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984fe911-653e-480c-99b4-307284c06b87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:09.312563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984fe911-653e-480c-99b4-307284c06b87 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:09.330548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:09.339626 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 351/686] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:38:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:09.342272 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288f81b-4171-4d73-a7bb-a72fa019f094 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:09.342772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288f81b-4171-4d73-a7bb-a72fa019f094 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:09.343420 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288f81b-4171-4d73-a7bb-a72fa019f094 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:09.353179 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d793b-d5a6-408a-827f-d2039b13cd74 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:09.353397 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 352/687] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:09 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:09.355150 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1288f81b-4171-4d73-a7bb-a72fa019f094 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:09.392152 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b1598a-bca6-4218-844b-7c4a6c059c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:09.392790 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1598a-bca6-4218-844b-7c4a6c059c68 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:09.393068 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1598a-bca6-4218-844b-7c4a6c059c68 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:09.393340 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1598a-bca6-4218-844b-7c4a6c059c68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:09.405927 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b1598a-bca6-4218-844b-7c4a6c059c68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:09.408405 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b1598a-bca6-4218-844b-7c4a6c059c68 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:09.431770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-55b1598a-bca6-4218-844b-7c4a6c059c68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:09.431770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:09.431770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:09.431770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:09.431770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:09.431770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:09.431770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:09.431770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:09.431770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:09.431770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:09.431770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:09.431770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:09.431770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:09.431770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:09.431770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:09.431770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:09.433224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:09.433224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:09.433224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:09.433224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:09.433224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:09.433224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:09.433224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:09.433224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:09.433224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:09.433224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:09.433224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:09.433224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:09.433224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:09.433224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:09.433224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:09.433224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:09.433224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:09.434098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:09.434098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:09.434098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:09.434098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 336/688] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:09.436415 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466a5af-2314-44f6-9def-b771cd564fd7 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:09.436415 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466a5af-2314-44f6-9def-b771cd564fd7 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:09.436777 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466a5af-2314-44f6-9def-b771cd564fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:09.469938 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-9466a5af-2314-44f6-9def-b771cd564fd7 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:09.472033 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9466a5af-2314-44f6-9def-b771cd564fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:09.508184 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1288f81b-4171-4d73-a7bb-a72fa019f094 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:09.510940 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:09.559731 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc96d6a-0e76-4ab9-ac7b-b421b4ed2390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:09.560711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc96d6a-0e76-4ab9-ac7b-b421b4ed2390 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:09.561163 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc96d6a-0e76-4ab9-ac7b-b421b4ed2390 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:09.561546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc96d6a-0e76-4ab9-ac7b-b421b4ed2390 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:09.562695 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc96d6a-0e76-4ab9-ac7b-b421b4ed2390 None admin] RBAC: Adding query filter params (role.id=33ac296b51c143e7bb44211175752edb, scope.project.id=543d44293d894a7fbb83401f7a26dd9b, user.id=997b4ffc01fc467ebbd1092787f217f6) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:09.563271 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc96d6a-0e76-4ab9-ac7b-b421b4ed2390 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:09.565643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc96d6a-0e76-4ab9-ac7b-b421b4ed2390 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:09.576279 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 353/689] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:38:09 2022] GET /identity/v3/role_assignments?user.id=997b4ffc01fc467ebbd1092787f217f6&scope.project.id=543d44293d894a7fbb83401f7a26dd9b&role.id=33ac296b51c143e7bb44211175752edb => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:09.577498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc96d6a-0e76-4ab9-ac7b-b421b4ed2390 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:09.600469 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 337/690] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:38:09.601947 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9466a5af-2314-44f6-9def-b771cd564fd7 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:09.773669 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc96d6a-0e76-4ab9-ac7b-b421b4ed2390 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:09.799065 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:09.817743 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ed842-784d-4c19-bb86-0dda1243dfbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:09.818503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ed842-784d-4c19-bb86-0dda1243dfbc None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:09.818788 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ed842-784d-4c19-bb86-0dda1243dfbc None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:09.819107 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ed842-784d-4c19-bb86-0dda1243dfbc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:09.831627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ed842-784d-4c19-bb86-0dda1243dfbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:38:09.838421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-113ed842-784d-4c19-bb86-0dda1243dfbc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:38:09.838421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:09.838421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:09.838421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:09.838421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:09.838421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:09.838421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:09.838421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:09.838421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:09.838421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:09.838421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:09.838421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:09.838421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:38:09.838421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:38:09.838421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:38:09.838421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:38:09.839463 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:38:09.839463 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:38:09.839463 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:38:09.839463 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:38:09.839463 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:09.839463 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:09.839463 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:09.839463 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:09.839463 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:09.839463 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:09.839463 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:09.839463 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:09.839463 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:09.839463 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:09.839463 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:09.839463 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:09.840302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:09.840302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:09.840302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:09.840302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:09.840302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:38:09.840302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:38:09.840302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:38:09.840302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:38:09.840302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:38:09.840302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:38:09.840302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:38:09.840302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:09.840302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 354/691] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:38:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:09.849027 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ed842-784d-4c19-bb86-0dda1243dfbc None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:09.861153 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9466a5af-2314-44f6-9def-b771cd564fd7 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:09.886092 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:09.942035 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6bc27-38f7-4e5a-9503-cc769f4813b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:09.944032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6bc27-38f7-4e5a-9503-cc769f4813b4 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:09.944747 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6bc27-38f7-4e5a-9503-cc769f4813b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:09.945431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6bc27-38f7-4e5a-9503-cc769f4813b4 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:09.946958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d6bc27-38f7-4e5a-9503-cc769f4813b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:09.948216 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d6bc27-38f7-4e5a-9503-cc769f4813b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:09.953347 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d6bc27-38f7-4e5a-9503-cc769f4813b4 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:09.980725 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 338/692] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:09.993197 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6bc27-38f7-4e5a-9503-cc769f4813b4 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:10.058278 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ed842-784d-4c19-bb86-0dda1243dfbc None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:10.115835 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca2bf2-1b28-46fc-ae4e-24f1ea3739f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:10.116974 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca2bf2-1b28-46fc-ae4e-24f1ea3739f8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:10.117327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca2bf2-1b28-46fc-ae4e-24f1ea3739f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:10.122950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca2bf2-1b28-46fc-ae4e-24f1ea3739f8 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:10.122950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca2bf2-1b28-46fc-ae4e-24f1ea3739f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:10.122950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca2bf2-1b28-46fc-ae4e-24f1ea3739f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:10.122950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca2bf2-1b28-46fc-ae4e-24f1ea3739f8 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:10.139044 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 355/693] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:38:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:10.140188 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca2bf2-1b28-46fc-ae4e-24f1ea3739f8 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:10.200151 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6bc27-38f7-4e5a-9503-cc769f4813b4 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:10.237861 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:10.256570 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d44a6a-886a-4d4f-8f85-cd57c77268e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:10.258080 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d44a6a-886a-4d4f-8f85-cd57c77268e3 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:10.258555 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d44a6a-886a-4d4f-8f85-cd57c77268e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:10.259121 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d44a6a-886a-4d4f-8f85-cd57c77268e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:10.267716 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d44a6a-886a-4d4f-8f85-cd57c77268e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:10.269457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d44a6a-886a-4d4f-8f85-cd57c77268e3 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:10.276271 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:10.284148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-c1d44a6a-886a-4d4f-8f85-cd57c77268e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:10.284148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:10.284148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:10.284148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:10.284148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:10.284148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:10.284148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:10.284148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:10.284148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:10.284148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:10.284148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:10.284148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:10.284148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:38:10.284148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:38:10.284148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:38:10.285965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:38:10.285965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:10.285965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:10.285965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:10.285965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:10.285965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:10.285965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:10.285965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:10.285965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:10.285965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:10.285965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:10.285965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:10.285965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:10.285965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:10.285965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:10.285965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:10.285965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:10.287920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:38:10.287920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:38:10.287920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:38:10.287920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:38:10.287920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:38:10.287920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:38:10.287920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:10.287920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:10.290167 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 339/694] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:38:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:38:10.291301 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d44a6a-886a-4d4f-8f85-cd57c77268e3 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:10.293923 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca2bf2-1b28-46fc-ae4e-24f1ea3739f8 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:10.344413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35381a-99d0-4302-9140-39d2f7ae7066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:10.345076 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35381a-99d0-4302-9140-39d2f7ae7066 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:10.345440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35381a-99d0-4302-9140-39d2f7ae7066 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:10.345799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35381a-99d0-4302-9140-39d2f7ae7066 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:10.359188 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e35381a-99d0-4302-9140-39d2f7ae7066 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:10.360755 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e35381a-99d0-4302-9140-39d2f7ae7066 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:10.377041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-2e35381a-99d0-4302-9140-39d2f7ae7066 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:10.377041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:10.377041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:10.377041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:10.377041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:10.377041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:10.377041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:10.377041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:10.377041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:10.377041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:10.377041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:10.377041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:10.377041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:10.377041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:10.377041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:10.377041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:10.378067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:10.378067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:10.378067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:10.378067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:10.378067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:10.378067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:10.378067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:10.378067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:10.378067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:10.378067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:10.378067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:10.378067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:10.378067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:10.378067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:10.378067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:10.378067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:10.378067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:10.379017 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:10.379017 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:10.379017 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:10.379017 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 356/695] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:10.379776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35381a-99d0-4302-9140-39d2f7ae7066 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:10.441648 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d44a6a-886a-4d4f-8f85-cd57c77268e3 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:10.482189 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0c498-2f90-4ee7-9ac0-93e885eea0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:10.483381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0c498-2f90-4ee7-9ac0-93e885eea0fd None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:10.483876 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0c498-2f90-4ee7-9ac0-93e885eea0fd None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:10.484389 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0c498-2f90-4ee7-9ac0-93e885eea0fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:10.497866 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0c498-2f90-4ee7-9ac0-93e885eea0fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:10.499812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0c498-2f90-4ee7-9ac0-93e885eea0fd None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:10.516457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-34c0c498-2f90-4ee7-9ac0-93e885eea0fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:10.516457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:10.516457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:10.516457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:10.516457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:10.516457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:10.516457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:10.516457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:10.516457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:10.516457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:10.516457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:10.516457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:10.516457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:10.516457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:10.516457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:10.516457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:10.517511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:10.517511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:10.517511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:10.517511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:10.517511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:10.517511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:10.517511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:10.517511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:10.517511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:10.517511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:10.517511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:10.517511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:10.517511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:10.517511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:10.517511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:10.517511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:10.517511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:10.518466 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:10.518466 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:10.518466 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:10.518660 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35381a-99d0-4302-9140-39d2f7ae7066 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:10.520946 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 340/696] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:10.522241 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0c498-2f90-4ee7-9ac0-93e885eea0fd None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:10.571678 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:10.582606 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44315539-2915-4fdc-bd62-58eacb24bdea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:10.583917 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44315539-2915-4fdc-bd62-58eacb24bdea None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:10.584370 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44315539-2915-4fdc-bd62-58eacb24bdea None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:10.585020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44315539-2915-4fdc-bd62-58eacb24bdea None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:10.586140 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44315539-2915-4fdc-bd62-58eacb24bdea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:10.586753 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44315539-2915-4fdc-bd62-58eacb24bdea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:10.589699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44315539-2915-4fdc-bd62-58eacb24bdea None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:10.613033 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 357/697] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:38:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:10.614479 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44315539-2915-4fdc-bd62-58eacb24bdea None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:10.621551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:10.664864 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0c498-2f90-4ee7-9ac0-93e885eea0fd None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:10.707115 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d31846-b968-41bb-9f48-0a2950aea5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:10.707901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d31846-b968-41bb-9f48-0a2950aea5ab None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:10.708301 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d31846-b968-41bb-9f48-0a2950aea5ab None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:10.708727 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d31846-b968-41bb-9f48-0a2950aea5ab None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:10.709563 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d31846-b968-41bb-9f48-0a2950aea5ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:10.709967 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d31846-b968-41bb-9f48-0a2950aea5ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:10.711845 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d31846-b968-41bb-9f48-0a2950aea5ab None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:10.731533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 341/698] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:10.733292 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d31846-b968-41bb-9f48-0a2950aea5ab None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:10.755782 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44315539-2915-4fdc-bd62-58eacb24bdea None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:10.802202 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c66c51-2df2-4a17-9523-811feaa83081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:10.803314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c66c51-2df2-4a17-9523-811feaa83081 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:10.803839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c66c51-2df2-4a17-9523-811feaa83081 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:10.804311 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c66c51-2df2-4a17-9523-811feaa83081 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:10.805230 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c66c51-2df2-4a17-9523-811feaa83081 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:10.805728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c66c51-2df2-4a17-9523-811feaa83081 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:10.808037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c66c51-2df2-4a17-9523-811feaa83081 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:10.825949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 358/699] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:10.827235 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c66c51-2df2-4a17-9523-811feaa83081 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:10.856659 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:10.867575 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d31846-b968-41bb-9f48-0a2950aea5ab None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:10.913862 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:10.931151 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468d2e8-ea63-4872-81bf-ac2bf2fa5b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:10.932175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468d2e8-ea63-4872-81bf-ac2bf2fa5b1a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:10.932646 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468d2e8-ea63-4872-81bf-ac2bf2fa5b1a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:10.933130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468d2e8-ea63-4872-81bf-ac2bf2fa5b1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:10.934130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468d2e8-ea63-4872-81bf-ac2bf2fa5b1a None admin] RBAC: Adding query filter params (role.id=33ac296b51c143e7bb44211175752edb, scope.project.id=543d44293d894a7fbb83401f7a26dd9b, user.id=1723e961375b45b7b3bd99ddeb69683f) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:10.934687 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468d2e8-ea63-4872-81bf-ac2bf2fa5b1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:10.936700 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468d2e8-ea63-4872-81bf-ac2bf2fa5b1a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:10.959583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 342/700] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:38:10 2022] GET /identity/v3/role_assignments?user.id=1723e961375b45b7b3bd99ddeb69683f&scope.project.id=543d44293d894a7fbb83401f7a26dd9b&role.id=33ac296b51c143e7bb44211175752edb => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:10.959583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468d2e8-ea63-4872-81bf-ac2bf2fa5b1a None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:10.983935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c66c51-2df2-4a17-9523-811feaa83081 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:11.045057 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b30dfd-f240-40de-92e9-df3101f09875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:11.045861 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b30dfd-f240-40de-92e9-df3101f09875 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:11.046140 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b30dfd-f240-40de-92e9-df3101f09875 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:11.046403 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b30dfd-f240-40de-92e9-df3101f09875 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:11.047209 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b30dfd-f240-40de-92e9-df3101f09875 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:11.049642 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b30dfd-f240-40de-92e9-df3101f09875 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:11.087539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 359/701] 10.222.0.71 () {60 vars in 1132 bytes} [Fri Aug 26 08:38:10 2022] POST /identity/v3/users => generated 254 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:38:11.194823 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468d2e8-ea63-4872-81bf-ac2bf2fa5b1a None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:11.241492 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c228164-a431-49f1-86b4-616089b6da99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:11.242143 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c228164-a431-49f1-86b4-616089b6da99 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:11.242428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c228164-a431-49f1-86b4-616089b6da99 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:11.242701 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c228164-a431-49f1-86b4-616089b6da99 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:11.258874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c228164-a431-49f1-86b4-616089b6da99 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:11.261638 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c228164-a431-49f1-86b4-616089b6da99 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:11.276428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-5c228164-a431-49f1-86b4-616089b6da99 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 08:38:11.276428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:11.276428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:11.276428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:11.276428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:11.276428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:11.276428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:11.276428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:11.276428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:11.276428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:11.276428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:11.276428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:11.276428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:38:11.276428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:38:11.276428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:38:11.276428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:38:11.278343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:11.278343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:11.278343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:38:11.278343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:11.278343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:38:11.278343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:11.278343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:11.278343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:11.278343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:11.278343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:11.278343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:11.278343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:11.278343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:11.278343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:11.278343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:11.278343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:11.278343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:11.278343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:11.281094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:11.281094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:11.281094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:38:11.281094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:38:11.281094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:38:11.281094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:38:11.281094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:38:11.281094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:38:11.281094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 08:38:11.281094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:11.281094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 343/702] 10.222.0.71 () {56 vars in 1093 bytes} [Fri Aug 26 08:38:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:11.287027 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b30dfd-f240-40de-92e9-df3101f09875 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:11.478108 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b30dfd-f240-40de-92e9-df3101f09875 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:11.501367 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:11.528237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c75db-38e9-4886-a72d-dc599f7e50c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:11.528950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c75db-38e9-4886-a72d-dc599f7e50c5 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:11.529322 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c75db-38e9-4886-a72d-dc599f7e50c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:11.529633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c75db-38e9-4886-a72d-dc599f7e50c5 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:11.530395 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c75db-38e9-4886-a72d-dc599f7e50c5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:11.530701 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c75db-38e9-4886-a72d-dc599f7e50c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:11.532419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c75db-38e9-4886-a72d-dc599f7e50c5 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:11.561762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 360/703] 10.222.0.71 () {56 vars in 1133 bytes} [Fri Aug 26 08:38:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:11.568887 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c228164-a431-49f1-86b4-616089b6da99 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:11.608891 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:11.712849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b576827-c1e1-4329-a6fe-d168f1e909c1 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:11.713134 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b576827-c1e1-4329-a6fe-d168f1e909c1 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:11.713417 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b576827-c1e1-4329-a6fe-d168f1e909c1 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:11.714859 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 361/704] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:38:11.723219 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e191ea3-9876-4e21-a68c-b48425201f55 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:11.723487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e191ea3-9876-4e21-a68c-b48425201f55 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:11.723906 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e191ea3-9876-4e21-a68c-b48425201f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:11.748345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-0e191ea3-9876-4e21-a68c-b48425201f55 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:11.749476 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-0e191ea3-9876-4e21-a68c-b48425201f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:11.784616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c228164-a431-49f1-86b4-616089b6da99 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:11.865773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821fd241-b0f4-4985-8aa4-d5912666d59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:11.866692 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821fd241-b0f4-4985-8aa4-d5912666d59d None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:11.867156 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821fd241-b0f4-4985-8aa4-d5912666d59d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:11.867676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821fd241-b0f4-4985-8aa4-d5912666d59d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:11.910373 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 362/705] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 08:38:12.019384 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821fd241-b0f4-4985-8aa4-d5912666d59d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:12.020848 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821fd241-b0f4-4985-8aa4-d5912666d59d None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:12.052045 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:12.203856 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 344/706] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:38:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 08:38:12.210222 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e191ea3-9876-4e21-a68c-b48425201f55 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:12.280235 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821fd241-b0f4-4985-8aa4-d5912666d59d None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:12.437812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e191ea3-9876-4e21-a68c-b48425201f55 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:12.441183 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821fd241-b0f4-4985-8aa4-d5912666d59d None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:12.447402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:12.475424 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:12.492877 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7b204-1863-44df-9956-ab2097673a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:12.492877 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7b204-1863-44df-9956-ab2097673a70 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:12.492877 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7b204-1863-44df-9956-ab2097673a70 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:12.492877 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7b204-1863-44df-9956-ab2097673a70 None admin] PATH_INFO: `/v3/services/network` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:12.493969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c7b204-1863-44df-9956-ab2097673a70 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:12.496454 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c7b204-1863-44df-9956-ab2097673a70 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:12.510773 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09eb86-c0c7-4c2e-95f7-25fcc0b9ae43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:12.511799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09eb86-c0c7-4c2e-95f7-25fcc0b9ae43 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:12.512256 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09eb86-c0c7-4c2e-95f7-25fcc0b9ae43 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:12.512702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09eb86-c0c7-4c2e-95f7-25fcc0b9ae43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:12.514406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-11c7b204-1863-44df-9956-ab2097673a70 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 08:38:12.514406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:12.514406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 08:38:12.514406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 08:38:12.514406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 08:38:12.514406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 08:38:12.514406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 08:38:12.514406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:12.514406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 08:38:12.514406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:12.514406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:38:12.514406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:12.514406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:12.514406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:12.514406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:12.514406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:12.514406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:12.515729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:12.515729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:12.515729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:12.515729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:12.515729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:12.515729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:12.515729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 08:38:12.515729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 08:38:12.515729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 08:38:12.515729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 08:38:12.515729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:12.515729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:12.515729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:12.515729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:12.515729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:12.515729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:12.522734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:12.522734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:12.522734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:12.522734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:12.522734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:12.522734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:12.522734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:12.522734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:12.522734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:12.522734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:12.522734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 08:38:12.522734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:12.522734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 08:38:12.522734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:12.522734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 345/707] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:38:12 2022] GET /identity/v3/services/network => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:38:12.524043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad53015-624e-48a6-9290-3da431c7e283 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:12.524043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad53015-624e-48a6-9290-3da431c7e283 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:12.524043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad53015-624e-48a6-9290-3da431c7e283 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:12.524043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 346/708] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:12.536025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad53015-624e-48a6-9290-3da431c7e283 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:12.536773 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09eb86-c0c7-4c2e-95f7-25fcc0b9ae43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:12.539651 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09eb86-c0c7-4c2e-95f7-25fcc0b9ae43 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:12.560728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-6c09eb86-c0c7-4c2e-95f7-25fcc0b9ae43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:12.560728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:12.560728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:12.560728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:12.560728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:12.560728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:12.560728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:12.560728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:12.560728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:12.560728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:12.560728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:12.560728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:12.560728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:12.560728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:12.560728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:12.560728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:12.561982 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:12.561982 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:12.561982 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:12.561982 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:12.561982 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:12.561982 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:12.561982 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:12.561982 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:12.561982 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:12.561982 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:12.561982 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:12.561982 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:12.561982 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:12.561982 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:12.561982 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:12.561982 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:12.561982 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:12.563122 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:12.563122 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:12.563122 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:12.568852 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 363/709] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:12.568852 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd2045-47a3-4db8-a445-0728ecb323af None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:12.568852 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd2045-47a3-4db8-a445-0728ecb323af None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:12.568852 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd2045-47a3-4db8-a445-0728ecb323af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:12.588025 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-1bcd2045-47a3-4db8-a445-0728ecb323af None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:12.590499 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1bcd2045-47a3-4db8-a445-0728ecb323af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:12.664150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad53015-624e-48a6-9290-3da431c7e283 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:12.681638 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 364/710] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:38:12.683031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd2045-47a3-4db8-a445-0728ecb323af None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:12.735810 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdcf1a-a66c-482f-92b0-57a55cfe4527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:12.736480 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdcf1a-a66c-482f-92b0-57a55cfe4527 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:12.736757 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdcf1a-a66c-482f-92b0-57a55cfe4527 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:12.737030 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdcf1a-a66c-482f-92b0-57a55cfe4527 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:12.737686 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfdcf1a-a66c-482f-92b0-57a55cfe4527 None admin] RBAC: Adding query filter params (name=network) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:12.737985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfdcf1a-a66c-482f-92b0-57a55cfe4527 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:12.739454 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfdcf1a-a66c-482f-92b0-57a55cfe4527 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:12.747652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 347/711] 10.222.0.71 () {56 vars in 1112 bytes} [Fri Aug 26 08:38:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:12.753848 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdcf1a-a66c-482f-92b0-57a55cfe4527 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:12.812315 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:12.839635 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:12.843606 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd2045-47a3-4db8-a445-0728ecb323af None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:12.911097 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdcf1a-a66c-482f-92b0-57a55cfe4527 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:12.918110 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc7a76-655a-46c7-b44f-94571f8c10e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:12.919358 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc7a76-655a-46c7-b44f-94571f8c10e9 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:12.919972 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc7a76-655a-46c7-b44f-94571f8c10e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:12.920655 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc7a76-655a-46c7-b44f-94571f8c10e9 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:12.929776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc7a76-655a-46c7-b44f-94571f8c10e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:12.930321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc7a76-655a-46c7-b44f-94571f8c10e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:12.931909 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc7a76-655a-46c7-b44f-94571f8c10e9 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:12.954150 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 365/712] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:12.962621 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc7a76-655a-46c7-b44f-94571f8c10e9 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:13.035814 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f8e9d-af67-4926-9c75-2d068c411548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:13.038211 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f8e9d-af67-4926-9c75-2d068c411548 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:13.042305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f8e9d-af67-4926-9c75-2d068c411548 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:13.043010 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f8e9d-af67-4926-9c75-2d068c411548 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:13.044444 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8f8e9d-af67-4926-9c75-2d068c411548 None admin] RBAC: Adding query filter params (type=network) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:13.044988 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8f8e9d-af67-4926-9c75-2d068c411548 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:13.046647 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8f8e9d-af67-4926-9c75-2d068c411548 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:13.063419 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 348/713] 10.222.0.71 () {56 vars in 1112 bytes} [Fri Aug 26 08:38:12 2022] GET /identity/v3/services?type=network => generated 124 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:13.158215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc7a76-655a-46c7-b44f-94571f8c10e9 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:13.173527 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f8e9d-af67-4926-9c75-2d068c411548 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:13.222993 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc68ed9-bcec-4537-b004-33e555cb5048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:13.223666 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc68ed9-bcec-4537-b004-33e555cb5048 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:13.223978 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc68ed9-bcec-4537-b004-33e555cb5048 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:13.224251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc68ed9-bcec-4537-b004-33e555cb5048 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:13.238749 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc68ed9-bcec-4537-b004-33e555cb5048 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:13.240381 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc68ed9-bcec-4537-b004-33e555cb5048 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:13.265389 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-dcc68ed9-bcec-4537-b004-33e555cb5048 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:13.265389 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:13.265389 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:13.265389 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:13.265389 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:13.265389 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:13.265389 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:13.265389 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:13.265389 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:13.265389 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:13.265389 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:13.265389 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:13.265389 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:38:13.265389 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:38:13.265389 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:38:13.266564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:38:13.266564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:13.266564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:13.266564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:13.266564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:13.266564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:13.266564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:13.266564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:13.266564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:13.266564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:13.266564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:13.266564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:13.266564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:13.266564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:13.266564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:13.266564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:13.266564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:13.267513 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:38:13.267513 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:38:13.267513 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:38:13.267513 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:38:13.267513 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:38:13.267513 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:38:13.267513 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:13.267513 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:13.267513 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 366/714] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:38:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:13.280072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc68ed9-bcec-4537-b004-33e555cb5048 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:13.311282 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:13.390958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f8e9d-af67-4926-9c75-2d068c411548 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:13.414816 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:13.456720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb5836-43e9-41c2-a191-baf3c8b750dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:13.457881 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb5836-43e9-41c2-a191-baf3c8b750dc None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:13.458456 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb5836-43e9-41c2-a191-baf3c8b750dc None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:13.459026 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb5836-43e9-41c2-a191-baf3c8b750dc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:13.460305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb5836-43e9-41c2-a191-baf3c8b750dc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:13.462708 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb5836-43e9-41c2-a191-baf3c8b750dc None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:13.473098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-fbdb5836-43e9-41c2-a191-baf3c8b750dc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 08:38:13.473098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:13.473098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 08:38:13.473098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 08:38:13.473098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 08:38:13.473098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 08:38:13.473098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 08:38:13.473098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:13.473098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 08:38:13.473098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:13.473098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:38:13.473098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:13.473098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:13.473098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:13.473098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:13.473098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:13.473098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:13.474531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:13.474531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:13.474531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:13.474531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:13.474531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:13.474531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:13.474531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 08:38:13.474531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 08:38:13.474531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 08:38:13.474531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 08:38:13.474531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:13.474531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:13.474531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:13.474531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:13.474531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:13.474531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:13.475399 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:13.475399 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:13.475399 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:13.475399 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:13.475399 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:13.475399 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:13.475399 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:13.475399 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:13.475399 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:13.475399 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:13.475399 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 08:38:13.475399 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:13.475399 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 08:38:13.475399 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:13.479358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 349/715] 10.222.0.71 () {56 vars in 1129 bytes} [Fri Aug 26 08:38:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:13.483684 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb5836-43e9-41c2-a191-baf3c8b750dc None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:13.498506 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc68ed9-bcec-4537-b004-33e555cb5048 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:13.540961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1659d48-814f-4915-94c1-ac197db61c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:13.542020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1659d48-814f-4915-94c1-ac197db61c19 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:13.542477 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1659d48-814f-4915-94c1-ac197db61c19 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:13.542957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1659d48-814f-4915-94c1-ac197db61c19 None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:13.544021 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1659d48-814f-4915-94c1-ac197db61c19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:13.544761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1659d48-814f-4915-94c1-ac197db61c19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:13.547045 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1659d48-814f-4915-94c1-ac197db61c19 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:13.566112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 367/716] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:38:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:13.573142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1659d48-814f-4915-94c1-ac197db61c19 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:13.616992 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb5836-43e9-41c2-a191-baf3c8b750dc None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:13.659102 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63696c96-954b-44c4-969f-51c093035f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:13.659720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63696c96-954b-44c4-969f-51c093035f8d None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:13.660118 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63696c96-954b-44c4-969f-51c093035f8d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:13.660381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63696c96-954b-44c4-969f-51c093035f8d None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:13.661199 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63696c96-954b-44c4-969f-51c093035f8d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:13.661851 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63696c96-954b-44c4-969f-51c093035f8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:13.663986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63696c96-954b-44c4-969f-51c093035f8d None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:13.673856 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 350/717] 10.222.0.71 () {56 vars in 1124 bytes} [Fri Aug 26 08:38:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:13.675344 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63696c96-954b-44c4-969f-51c093035f8d None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:13.681729 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:13.734077 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:13.763689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1659d48-814f-4915-94c1-ac197db61c19 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:13.878687 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea11d75-caf7-48ca-9935-4ef6b1cd22b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:13.879350 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea11d75-caf7-48ca-9935-4ef6b1cd22b6 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:13.879630 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea11d75-caf7-48ca-9935-4ef6b1cd22b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:13.879902 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea11d75-caf7-48ca-9935-4ef6b1cd22b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:13.882059 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea11d75-caf7-48ca-9935-4ef6b1cd22b6 None admin] RBAC: Adding query filter params (role.id=18c3c8a821b647f7982a955880bd0a9a, scope.project.id=543d44293d894a7fbb83401f7a26dd9b, user.id=f16d1768715a41648af3fe6afadd7477) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:13.883950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea11d75-caf7-48ca-9935-4ef6b1cd22b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:13.887459 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea11d75-caf7-48ca-9935-4ef6b1cd22b6 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:13.891510 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63696c96-954b-44c4-969f-51c093035f8d None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:13.911172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 368/718] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:38:13 2022] GET /identity/v3/role_assignments?user.id=f16d1768715a41648af3fe6afadd7477&scope.project.id=543d44293d894a7fbb83401f7a26dd9b&role.id=18c3c8a821b647f7982a955880bd0a9a => generated 259 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:13.953920 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4728d-c642-4002-b8bf-c7d3b89946ca None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:13.954345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4728d-c642-4002-b8bf-c7d3b89946ca None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:13.954490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4728d-c642-4002-b8bf-c7d3b89946ca None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:13.956652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 369/719] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:38:13.966435 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07781cf9-d6d9-422e-bf64-104f458f53c6 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:13.966821 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07781cf9-d6d9-422e-bf64-104f458f53c6 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:13.967639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07781cf9-d6d9-422e-bf64-104f458f53c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:13.990488 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f2330-3b2c-4b98-af09-c335e172399c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:13.991195 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f2330-3b2c-4b98-af09-c335e172399c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:13.991470 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f2330-3b2c-4b98-af09-c335e172399c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:13.991739 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f2330-3b2c-4b98-af09-c335e172399c None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:13.992400 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f2330-3b2c-4b98-af09-c335e172399c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:13.992702 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f2330-3b2c-4b98-af09-c335e172399c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:13.994460 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928f2330-3b2c-4b98-af09-c335e172399c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:14.003282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 351/720] 10.222.0.71 () {56 vars in 1124 bytes} [Fri Aug 26 08:38:13 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:14.004593 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-07781cf9-d6d9-422e-bf64-104f458f53c6 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:14.004767 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e961df6-ceab-4e1a-b016-5a328986a376 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:14.005093 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e961df6-ceab-4e1a-b016-5a328986a376 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:14.005369 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e961df6-ceab-4e1a-b016-5a328986a376 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:14.005880 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-07781cf9-d6d9-422e-bf64-104f458f53c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:14.007066 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 352/721] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:14.012969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615df553-56cd-4cb6-a3c1-db5887141c0c None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:14.013456 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615df553-56cd-4cb6-a3c1-db5887141c0c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:14.013577 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615df553-56cd-4cb6-a3c1-db5887141c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:14.037481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-615df553-56cd-4cb6-a3c1-db5887141c0c None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:14.039013 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-615df553-56cd-4cb6-a3c1-db5887141c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:14.144013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 370/722] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 08:38:14.189949 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 353/723] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 08:38:14.465609 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07781cf9-d6d9-422e-bf64-104f458f53c6 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:14.541326 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:14.562490 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615df553-56cd-4cb6-a3c1-db5887141c0c None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:14.621464 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:14.642861 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07781cf9-d6d9-422e-bf64-104f458f53c6 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:14.715516 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615df553-56cd-4cb6-a3c1-db5887141c0c None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:14.731763 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6eaf09-4112-4f30-8431-a37cd5b8c9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:14.732340 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6eaf09-4112-4f30-8431-a37cd5b8c9ee None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:14.732653 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6eaf09-4112-4f30-8431-a37cd5b8c9ee None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:14.732927 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6eaf09-4112-4f30-8431-a37cd5b8c9ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:14.757023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6eaf09-4112-4f30-8431-a37cd5b8c9ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:38:14.758370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf8e82e-92a5-4aac-b4c9-48edab758658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:14.759546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8e82e-92a5-4aac-b4c9-48edab758658 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:14.759983 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8e82e-92a5-4aac-b4c9-48edab758658 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:14.760410 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8e82e-92a5-4aac-b4c9-48edab758658 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:14.773145 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf8e82e-92a5-4aac-b4c9-48edab758658 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:38:14.774243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-4f6eaf09-4112-4f30-8431-a37cd5b8c9ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:38:14.774243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:14.774243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:14.774243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:14.774243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:14.774243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:14.774243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:14.774243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:14.774243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:14.774243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:14.774243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:14.774243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:14.774243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:38:14.774243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:38:14.774243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:38:14.774243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:38:14.775245 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:38:14.775245 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:38:14.775245 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:38:14.775245 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:38:14.775245 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:14.775245 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:14.775245 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:14.775245 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:14.775245 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:14.775245 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:14.775245 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:14.775245 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:14.775245 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:14.775245 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:14.775245 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:14.775245 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:14.776118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:14.776118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:14.776118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:14.776118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:14.776118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:38:14.776118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:38:14.776118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:38:14.776118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:38:14.776118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:38:14.776118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:38:14.776118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:38:14.776118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:14.776118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 371/724] 10.222.0.71 () {56 vars in 1102 bytes} [Fri Aug 26 08:38:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:14.781354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-ccf8e82e-92a5-4aac-b4c9-48edab758658 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:38:14.781354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:14.781354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:14.781354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:14.781354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:14.781354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:14.781354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:14.781354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:14.781354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:14.781354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:14.781354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:14.781354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:14.781354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:38:14.781354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:38:14.781354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:38:14.781354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:38:14.782629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:38:14.782629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:38:14.782629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:38:14.782629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:38:14.782629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:14.782629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:14.782629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:14.782629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:14.782629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:14.782629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:14.782629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:14.782629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:14.782629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:14.782629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:14.782629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:14.782629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:14.783733 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:14.783733 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:14.783733 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:14.783733 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:14.783733 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:38:14.783733 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:38:14.783733 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:38:14.783733 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:38:14.783733 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:38:14.783733 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:38:14.783733 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:38:14.783733 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:14.784431 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6eaf09-4112-4f30-8431-a37cd5b8c9ee None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:14.786115 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 354/725] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:38:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:38:14.792092 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf8e82e-92a5-4aac-b4c9-48edab758658 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:14.899084 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:14.909167 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf8e82e-92a5-4aac-b4c9-48edab758658 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:14.949729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c7bfc-b777-426d-9437-200e9b8d326f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:14.950634 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c7bfc-b777-426d-9437-200e9b8d326f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:14.951102 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c7bfc-b777-426d-9437-200e9b8d326f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:14.951516 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c7bfc-b777-426d-9437-200e9b8d326f None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:14.952339 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c7bfc-b777-426d-9437-200e9b8d326f None admin] RBAC: Adding query filter params (name=admin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:14.952834 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c7bfc-b777-426d-9437-200e9b8d326f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:14.954378 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c7bfc-b777-426d-9437-200e9b8d326f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:14.964679 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 355/726] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:38:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:14.970652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c7bfc-b777-426d-9437-200e9b8d326f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:15.014803 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6eaf09-4112-4f30-8431-a37cd5b8c9ee None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:15.080050 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c7bfc-b777-426d-9437-200e9b8d326f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:15.100650 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:15.119240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6fdb0-f961-49ab-af11-e4ea8eb24f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:15.120259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6fdb0-f961-49ab-af11-e4ea8eb24f2b None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:15.120737 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6fdb0-f961-49ab-af11-e4ea8eb24f2b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:15.121199 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6fdb0-f961-49ab-af11-e4ea8eb24f2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:15.133567 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6fdb0-f961-49ab-af11-e4ea8eb24f2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:15.135546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6fdb0-f961-49ab-af11-e4ea8eb24f2b None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:15.151263 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-08c6fdb0-f961-49ab-af11-e4ea8eb24f2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:15.151263 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:15.151263 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:15.151263 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:15.151263 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:15.151263 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:15.151263 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:15.151263 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:15.151263 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:15.151263 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:15.151263 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:15.151263 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:15.151263 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:15.151263 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:15.151263 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:15.151263 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:15.152278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:15.152278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:15.152278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:15.152278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:15.152278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:15.152278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:15.152278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:15.152278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:15.152278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:15.152278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:15.152278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:15.152278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:15.152278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:15.152278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:15.152278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:15.152278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:15.152278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:15.153151 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:15.153151 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:15.153151 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:15.155359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 356/727] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:38:15.161127 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4bbf55-5796-451e-b520-6713a40067ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:15.161503 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6fdb0-f961-49ab-af11-e4ea8eb24f2b None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:15.162300 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4bbf55-5796-451e-b520-6713a40067ac None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:15.162792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4bbf55-5796-451e-b520-6713a40067ac None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:15.163331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4bbf55-5796-451e-b520-6713a40067ac None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:15.164580 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4bbf55-5796-451e-b520-6713a40067ac None admin] RBAC: Adding query filter params (name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:15.165129 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4bbf55-5796-451e-b520-6713a40067ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:15.168102 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4bbf55-5796-451e-b520-6713a40067ac None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:15.175652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:15.204520 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 372/728] 10.222.0.71 () {56 vars in 1103 bytes} [Fri Aug 26 08:38:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:15.211110 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4bbf55-5796-451e-b520-6713a40067ac None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:15.276164 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6fdb0-f961-49ab-af11-e4ea8eb24f2b None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:15.321279 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b163ef-0648-424c-94be-bb49ebff5022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:15.322354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b163ef-0648-424c-94be-bb49ebff5022 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:15.323906 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b163ef-0648-424c-94be-bb49ebff5022 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:15.324575 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b163ef-0648-424c-94be-bb49ebff5022 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:15.326584 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b163ef-0648-424c-94be-bb49ebff5022 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:15.327125 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b163ef-0648-424c-94be-bb49ebff5022 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:15.328913 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b163ef-0648-424c-94be-bb49ebff5022 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:15.353443 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 357/729] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:15.359539 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b163ef-0648-424c-94be-bb49ebff5022 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:15.423638 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4bbf55-5796-451e-b520-6713a40067ac None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:15.492202 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f940bf-b86f-477f-a647-afe8024a1067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:15.496108 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f940bf-b86f-477f-a647-afe8024a1067 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:15.496690 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f940bf-b86f-477f-a647-afe8024a1067 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:15.497154 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f940bf-b86f-477f-a647-afe8024a1067 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:15.501120 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:15.525315 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f940bf-b86f-477f-a647-afe8024a1067 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:15.527269 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f940bf-b86f-477f-a647-afe8024a1067 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:15.559020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-39f940bf-b86f-477f-a647-afe8024a1067 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:15.559020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:15.559020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:15.559020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:15.559020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:15.559020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:15.559020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:15.559020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:15.559020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:15.559020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:15.559020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:15.559020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:15.559020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:15.559020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:15.559020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:15.559020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:15.560471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:15.560471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:15.560471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:15.560471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:15.560471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:15.560471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:15.560471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:15.560471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:15.560471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:15.560471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:15.560471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:15.560471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:15.560471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:15.560471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:15.560471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:15.560471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:15.560471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:15.561356 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:15.561356 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:15.561356 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:15.568565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 373/730] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:15.574794 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f940bf-b86f-477f-a647-afe8024a1067 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:15.580597 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b163ef-0648-424c-94be-bb49ebff5022 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:15.643249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bda69f-29ab-4c36-be62-c33da1c3badf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:15.644857 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bda69f-29ab-4c36-be62-c33da1c3badf None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:15.645512 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bda69f-29ab-4c36-be62-c33da1c3badf None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:15.648016 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bda69f-29ab-4c36-be62-c33da1c3badf None admin] PATH_INFO: `/v3/users/swift` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:15.651557 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:15.664034 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bda69f-29ab-4c36-be62-c33da1c3badf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:15.667186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bda69f-29ab-4c36-be62-c33da1c3badf None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:15.689982 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-87bda69f-29ab-4c36-be62-c33da1c3badf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 08:38:15.689982 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:15.689982 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:15.689982 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:15.689982 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:15.689982 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:15.689982 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:15.689982 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:15.689982 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:15.689982 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:15.689982 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:15.689982 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:15.689982 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:38:15.689982 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:38:15.689982 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:38:15.689982 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:38:15.691036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:15.691036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:15.691036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:38:15.691036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:15.691036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:38:15.691036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:15.691036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:15.691036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:15.691036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:15.691036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:15.691036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:15.691036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:15.691036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:15.691036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:15.691036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:15.691036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:15.691036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:15.691036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:15.691973 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:15.691973 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:15.691973 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:38:15.691973 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:38:15.691973 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:38:15.691973 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:38:15.691973 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:38:15.691973 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:38:15.691973 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 08:38:15.691973 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:15.695437 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 358/731] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:38:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:38:15.702851 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bda69f-29ab-4c36-be62-c33da1c3badf None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:15.817331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f940bf-b86f-477f-a647-afe8024a1067 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:15.830561 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bda69f-29ab-4c36-be62-c33da1c3badf None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:15.860170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:15.877986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7366433-b089-4e8d-bc2a-5f5b8b63a969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:15.879112 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7366433-b089-4e8d-bc2a-5f5b8b63a969 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:15.879694 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7366433-b089-4e8d-bc2a-5f5b8b63a969 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:15.880331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7366433-b089-4e8d-bc2a-5f5b8b63a969 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:15.881295 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7366433-b089-4e8d-bc2a-5f5b8b63a969 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:15.881878 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7366433-b089-4e8d-bc2a-5f5b8b63a969 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:15.882173 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50400942-63be-42f2-978e-e904e7ba3b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:15.882795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50400942-63be-42f2-978e-e904e7ba3b46 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:15.883182 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50400942-63be-42f2-978e-e904e7ba3b46 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:15.883461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50400942-63be-42f2-978e-e904e7ba3b46 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:15.884114 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50400942-63be-42f2-978e-e904e7ba3b46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:15.884420 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50400942-63be-42f2-978e-e904e7ba3b46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:15.884838 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7366433-b089-4e8d-bc2a-5f5b8b63a969 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:15.885786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50400942-63be-42f2-978e-e904e7ba3b46 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:15.899316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 374/732] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:15.908867 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50400942-63be-42f2-978e-e904e7ba3b46 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:15.915660 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 359/733] 10.222.0.71 () {56 vars in 1135 bytes} [Fri Aug 26 08:38:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:15.925174 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7366433-b089-4e8d-bc2a-5f5b8b63a969 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:16.067168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:16.072881 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7366433-b089-4e8d-bc2a-5f5b8b63a969 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:16.077563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50400942-63be-42f2-978e-e904e7ba3b46 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:16.110940 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea2d2e-4da4-4421-a310-42cd902d56a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:16.111659 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea2d2e-4da4-4421-a310-42cd902d56a0 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:16.112023 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea2d2e-4da4-4421-a310-42cd902d56a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:16.112455 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea2d2e-4da4-4421-a310-42cd902d56a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:16.127905 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c2957-a2a7-490c-a896-373f63a33bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:16.128889 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c2957-a2a7-490c-a896-373f63a33bd1 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:16.129580 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c2957-a2a7-490c-a896-373f63a33bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:16.130023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c2957-a2a7-490c-a896-373f63a33bd1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:16.143679 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28c2957-a2a7-490c-a896-373f63a33bd1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:16.146736 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28c2957-a2a7-490c-a896-373f63a33bd1 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:16.170148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-f28c2957-a2a7-490c-a896-373f63a33bd1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 08:38:16.170148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:16.170148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:16.170148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:16.170148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:16.170148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:16.170148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:16.170148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:16.170148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:16.170148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:16.170148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:16.170148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:16.170148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:38:16.170148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:38:16.170148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:38:16.170148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:38:16.171187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:16.171187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:16.171187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:38:16.171187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:16.171187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:38:16.171187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:16.171187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:16.171187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:16.171187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:16.171187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:16.171187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:16.171187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:16.171187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:16.171187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:16.171187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:16.171187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:16.171187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:16.171187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:16.172121 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:16.172121 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:16.172121 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:38:16.172121 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:38:16.172121 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:38:16.172121 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:38:16.172121 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:38:16.172121 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:38:16.172121 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 08:38:16.172121 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:16.176262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:16.179036 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 375/734] 10.222.0.71 () {56 vars in 1117 bytes} [Fri Aug 26 08:38:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:16.181509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57afd5-efaf-48f0-a186-da8270084e7f None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:16.181955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57afd5-efaf-48f0-a186-da8270084e7f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:16.182442 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff57afd5-efaf-48f0-a186-da8270084e7f None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:16.186394 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 376/735] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:16.187966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57afd5-efaf-48f0-a186-da8270084e7f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:16.193927 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea2d2e-4da4-4421-a310-42cd902d56a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:16.195847 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea2d2e-4da4-4421-a310-42cd902d56a0 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:16.307266 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 360/736] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:38:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 08:38:16.308781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c62d4-5fe6-4891-9929-c05cc190662b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:16.309266 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c62d4-5fe6-4891-9929-c05cc190662b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:16.309676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c62d4-5fe6-4891-9929-c05cc190662b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:16.329345 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-4f9c62d4-5fe6-4891-9929-c05cc190662b None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:16.330789 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4f9c62d4-5fe6-4891-9929-c05cc190662b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:16.380699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff57afd5-efaf-48f0-a186-da8270084e7f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:16.407436 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 361/737] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:38:16.408591 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9c62d4-5fe6-4891-9929-c05cc190662b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:16.428355 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:16.442165 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813422b2-05dd-4653-828a-805b2ab88cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:16.443243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813422b2-05dd-4653-828a-805b2ab88cf0 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:16.443722 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813422b2-05dd-4653-828a-805b2ab88cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:16.444177 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813422b2-05dd-4653-828a-805b2ab88cf0 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:16.445623 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813422b2-05dd-4653-828a-805b2ab88cf0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:16.446268 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813422b2-05dd-4653-828a-805b2ab88cf0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:16.448991 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813422b2-05dd-4653-828a-805b2ab88cf0 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:16.498831 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 377/738] 10.222.0.71 () {56 vars in 1149 bytes} [Fri Aug 26 08:38:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:16.501214 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813422b2-05dd-4653-828a-805b2ab88cf0 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:16.530903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9c62d4-5fe6-4891-9929-c05cc190662b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:16.537999 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:16.645436 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027adeea-c186-437e-9e13-2ea3addddaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:16.646515 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027adeea-c186-437e-9e13-2ea3addddaa4 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:16.647009 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027adeea-c186-437e-9e13-2ea3addddaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:16.647447 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027adeea-c186-437e-9e13-2ea3addddaa4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:16.668028 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027adeea-c186-437e-9e13-2ea3addddaa4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:16.669982 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027adeea-c186-437e-9e13-2ea3addddaa4 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:16.686626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-027adeea-c186-437e-9e13-2ea3addddaa4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:16.686626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:16.686626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:16.686626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:16.686626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:16.686626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:16.686626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:16.686626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:16.686626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:16.686626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:16.686626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:16.686626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:16.686626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:16.686626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:16.686626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:16.686626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:16.687657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:16.687657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:16.687657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:16.687657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:16.687657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:16.687657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:16.687657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:16.687657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:16.687657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:16.687657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:16.687657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:16.687657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:16.687657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:16.687657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:16.687657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:16.687657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:16.687657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:16.688543 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:16.688543 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:16.688543 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:16.690725 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 362/739] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:16.692168 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fab067-3f07-43dc-acda-d827ecff1c63 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:16.692668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fab067-3f07-43dc-acda-d827ecff1c63 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:16.693163 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fab067-3f07-43dc-acda-d827ecff1c63 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:16.694854 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 363/740] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:16.700127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e826fc-82e4-4745-ac4a-87a499db737d None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:16.700667 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e826fc-82e4-4745-ac4a-87a499db737d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:16.701107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e826fc-82e4-4745-ac4a-87a499db737d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:16.714753 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813422b2-05dd-4653-828a-805b2ab88cf0 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:16.726172 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-71e826fc-82e4-4745-ac4a-87a499db737d None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:16.727720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-71e826fc-82e4-4745-ac4a-87a499db737d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:16.776264 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727087b0-a7b4-4db1-96a6-d310174f9134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:16.776849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727087b0-a7b4-4db1-96a6-d310174f9134 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:16.777128 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727087b0-a7b4-4db1-96a6-d310174f9134 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:16.777389 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727087b0-a7b4-4db1-96a6-d310174f9134 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:16.814709 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:16.823181 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 364/741] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 08:38:16.823953 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e826fc-82e4-4745-ac4a-87a499db737d None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:16.890518 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727087b0-a7b4-4db1-96a6-d310174f9134 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:16.892381 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727087b0-a7b4-4db1-96a6-d310174f9134 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:16.908620 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:16.948400 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e826fc-82e4-4745-ac4a-87a499db737d None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:16.983834 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0305999-3df9-4e5c-b3d7-b381477def10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:16.984823 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0305999-3df9-4e5c-b3d7-b381477def10 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:16.984823 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0305999-3df9-4e5c-b3d7-b381477def10 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:16.985018 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0305999-3df9-4e5c-b3d7-b381477def10 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:16.985632 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0305999-3df9-4e5c-b3d7-b381477def10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:16.985932 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0305999-3df9-4e5c-b3d7-b381477def10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:16.987372 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0305999-3df9-4e5c-b3d7-b381477def10 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:17.004651 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 365/742] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:17.004651 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfaeb2-1c64-4437-8844-8f2811f54b86 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:17.004651 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfaeb2-1c64-4437-8844-8f2811f54b86 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:17.004651 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfaeb2-1c64-4437-8844-8f2811f54b86 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:17.004651 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 366/743] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:17.004651 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfaeb2-1c64-4437-8844-8f2811f54b86 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:17.021797 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 378/744] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:38:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 08:38:17.022680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727087b0-a7b4-4db1-96a6-d310174f9134 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:17.088578 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:17.140329 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfaeb2-1c64-4437-8844-8f2811f54b86 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:17.141899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727087b0-a7b4-4db1-96a6-d310174f9134 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:17.156813 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:17.180643 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4eb25a-c434-4fc1-8bf8-1746424294a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:17.181176 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4eb25a-c434-4fc1-8bf8-1746424294a0 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:17.181439 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4eb25a-c434-4fc1-8bf8-1746424294a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:17.181788 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4eb25a-c434-4fc1-8bf8-1746424294a0 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:17.181900 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0566df-d269-4e77-a5af-6d081654ddc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:17.182605 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4eb25a-c434-4fc1-8bf8-1746424294a0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:17.182714 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0566df-d269-4e77-a5af-6d081654ddc1 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:17.183017 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0566df-d269-4e77-a5af-6d081654ddc1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:17.183289 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0566df-d269-4e77-a5af-6d081654ddc1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:17.184025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4eb25a-c434-4fc1-8bf8-1746424294a0 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:17.192120 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0566df-d269-4e77-a5af-6d081654ddc1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:17.193655 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0566df-d269-4e77-a5af-6d081654ddc1 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:17.204572 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 367/745] 10.222.0.71 () {60 vars in 1141 bytes} [Fri Aug 26 08:38:17 2022] POST /identity/v3/services => generated 240 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:38:17.205820 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e848b-67d0-4432-b1c0-a1fc7591e704 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:17.206105 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e848b-67d0-4432-b1c0-a1fc7591e704 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:17.206376 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e848b-67d0-4432-b1c0-a1fc7591e704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:17.216770 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-af0566df-d269-4e77-a5af-6d081654ddc1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:17.216770 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:17.216770 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:17.216770 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:17.216770 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:17.216770 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:17.216770 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:17.216770 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:17.216770 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:17.216770 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:17.216770 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:17.216770 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:17.216770 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:38:17.216770 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:38:17.216770 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:38:17.217785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:38:17.217785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:17.217785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:17.217785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:17.217785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:17.217785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:17.217785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:17.217785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:17.217785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:17.217785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:17.217785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:17.217785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:17.217785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:17.217785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:17.217785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:17.217785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:17.217785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:17.218684 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:38:17.218684 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:38:17.218684 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:38:17.218684 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:38:17.218684 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:38:17.218684 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:38:17.218684 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:17.218684 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:17.218684 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 379/746] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:38:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:17.219312 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0566df-d269-4e77-a5af-6d081654ddc1 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:17.231747 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ec4e848b-67d0-4432-b1c0-a1fc7591e704 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:17.233078 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ec4e848b-67d0-4432-b1c0-a1fc7591e704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:17.323904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 368/747] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:17 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 08:38:17.324840 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e848b-67d0-4432-b1c0-a1fc7591e704 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:17.377436 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0566df-d269-4e77-a5af-6d081654ddc1 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:17.393707 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:17.425137 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79df17e-4111-4c4f-be5e-429023929885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:17.426194 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79df17e-4111-4c4f-be5e-429023929885 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:17.426194 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79df17e-4111-4c4f-be5e-429023929885 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:17.426463 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79df17e-4111-4c4f-be5e-429023929885 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:17.440688 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79df17e-4111-4c4f-be5e-429023929885 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:17.450959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79df17e-4111-4c4f-be5e-429023929885 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:17.462432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-f79df17e-4111-4c4f-be5e-429023929885 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:17.462432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:17.462432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:17.462432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:17.462432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:17.462432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:17.462432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:17.462432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:17.462432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:17.462432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:17.462432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:17.462432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:17.462432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:17.462432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:17.462432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:17.462432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:17.463503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:17.463503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:17.463503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:17.463503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:17.463503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:17.463503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:17.463503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:17.463503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:17.463503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:17.463503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:17.463503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:17.463503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:17.463503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:17.463503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:17.463503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:17.463503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:17.463503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:17.464434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:17.464434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:17.464434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:17.464605 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e848b-67d0-4432-b1c0-a1fc7591e704 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:17.464698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 380/748] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:17.465227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79df17e-4111-4c4f-be5e-429023929885 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:17.511296 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:17.524070 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c58b61f-add7-49ae-8fc8-bf612700aad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:17.524696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58b61f-add7-49ae-8fc8-bf612700aad9 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:17.525033 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58b61f-add7-49ae-8fc8-bf612700aad9 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:17.525309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58b61f-add7-49ae-8fc8-bf612700aad9 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:17.525969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c58b61f-add7-49ae-8fc8-bf612700aad9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:17.527395 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c58b61f-add7-49ae-8fc8-bf612700aad9 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:17.547488 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 369/749] 10.222.0.71 () {60 vars in 1141 bytes} [Fri Aug 26 08:38:17 2022] POST /identity/v3/services => generated 251 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:38:17.548424 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c58b61f-add7-49ae-8fc8-bf612700aad9 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:17.640379 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79df17e-4111-4c4f-be5e-429023929885 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:17.695425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6f488-4955-4ce3-90d4-d57f7e3dcdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:17.696030 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6f488-4955-4ce3-90d4-d57f7e3dcdb8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:17.696424 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6f488-4955-4ce3-90d4-d57f7e3dcdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:17.696711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6f488-4955-4ce3-90d4-d57f7e3dcdb8 None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:17.697452 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6f488-4955-4ce3-90d4-d57f7e3dcdb8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:17.697765 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6f488-4955-4ce3-90d4-d57f7e3dcdb8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:17.699280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6f488-4955-4ce3-90d4-d57f7e3dcdb8 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:17.727631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 381/750] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:38:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:17.728698 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c58b61f-add7-49ae-8fc8-bf612700aad9 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:17.728921 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6f488-4955-4ce3-90d4-d57f7e3dcdb8 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:17.732556 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:17.788275 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79373dda-6a62-49d7-8504-3679c157656e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:17.788904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79373dda-6a62-49d7-8504-3679c157656e None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:17.789180 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79373dda-6a62-49d7-8504-3679c157656e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:17.789452 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79373dda-6a62-49d7-8504-3679c157656e None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:17.790225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79373dda-6a62-49d7-8504-3679c157656e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:17.790543 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79373dda-6a62-49d7-8504-3679c157656e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:17.792060 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79373dda-6a62-49d7-8504-3679c157656e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:17.814315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 370/751] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:17.816135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79373dda-6a62-49d7-8504-3679c157656e None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:17.863845 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:17.886286 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6f488-4955-4ce3-90d4-d57f7e3dcdb8 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:17.935551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51e1eb-8e88-43a2-bf00-fcb8113092e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:17.936348 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51e1eb-8e88-43a2-bf00-fcb8113092e3 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:17.936721 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51e1eb-8e88-43a2-bf00-fcb8113092e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:17.937088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51e1eb-8e88-43a2-bf00-fcb8113092e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:17.952082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db51e1eb-8e88-43a2-bf00-fcb8113092e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:38:17.957730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-db51e1eb-8e88-43a2-bf00-fcb8113092e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:38:17.957730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:17.957730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:17.957730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:17.957730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:17.957730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:17.957730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:17.957730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:17.957730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:17.957730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:17.957730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:17.957730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:17.957730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:38:17.957730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:38:17.957730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:38:17.957730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:38:17.958724 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:38:17.958724 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:38:17.958724 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:38:17.958724 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:38:17.958724 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:17.958724 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:17.958724 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:17.958724 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:17.958724 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:17.958724 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:17.958724 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:17.958724 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:17.958724 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:17.958724 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:17.958724 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:17.958724 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:17.959606 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:17.959606 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:17.959606 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:17.959606 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:17.959606 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:38:17.959606 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:38:17.959606 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:38:17.959606 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:38:17.959606 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:38:17.959606 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:38:17.959606 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:38:17.959606 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:17.959606 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 382/752] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:38:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:17.960604 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51e1eb-8e88-43a2-bf00-fcb8113092e3 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:17.965632 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79373dda-6a62-49d7-8504-3679c157656e None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:18.005027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965cfd4b-eb49-4ea5-8f36-d264b471ecfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:18.005732 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965cfd4b-eb49-4ea5-8f36-d264b471ecfd None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:18.006005 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965cfd4b-eb49-4ea5-8f36-d264b471ecfd None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:18.006268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965cfd4b-eb49-4ea5-8f36-d264b471ecfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:18.007046 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965cfd4b-eb49-4ea5-8f36-d264b471ecfd None admin] RBAC: Adding query filter params (role.id=18c3c8a821b647f7982a955880bd0a9a, scope.project.id=543d44293d894a7fbb83401f7a26dd9b, user.id=1723e961375b45b7b3bd99ddeb69683f) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:18.007423 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965cfd4b-eb49-4ea5-8f36-d264b471ecfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:18.009581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965cfd4b-eb49-4ea5-8f36-d264b471ecfd None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:18.027032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 371/753] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:38:17 2022] GET /identity/v3/role_assignments?user.id=1723e961375b45b7b3bd99ddeb69683f&scope.project.id=543d44293d894a7fbb83401f7a26dd9b&role.id=18c3c8a821b647f7982a955880bd0a9a => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:18.028351 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965cfd4b-eb49-4ea5-8f36-d264b471ecfd None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:18.084277 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51e1eb-8e88-43a2-bf00-fcb8113092e3 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:18.086713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:18.131401 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a22b2d-ed47-4927-9b6f-280f4591b0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:18.132069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a22b2d-ed47-4927-9b6f-280f4591b0d5 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:18.132406 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a22b2d-ed47-4927-9b6f-280f4591b0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:18.132690 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a22b2d-ed47-4927-9b6f-280f4591b0d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:18.147351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a22b2d-ed47-4927-9b6f-280f4591b0d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:18.149853 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a22b2d-ed47-4927-9b6f-280f4591b0d5 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:18.164801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-e1a22b2d-ed47-4927-9b6f-280f4591b0d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:18.164801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:18.164801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:18.164801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:18.164801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:18.164801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:18.164801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:18.164801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:18.164801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:18.164801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:18.164801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:18.164801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:18.164801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:38:18.164801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:38:18.164801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:38:18.166865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:38:18.166865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:18.166865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:18.166865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:18.166865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:18.166865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:18.166865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:18.166865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:18.166865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:18.166865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:18.166865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:18.166865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:18.166865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:18.166865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:18.166865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:18.166865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:18.166865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:18.167799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:38:18.167799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:38:18.167799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:38:18.167799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:38:18.167799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:38:18.167799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:38:18.167799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:18.167799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:18.167799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 383/754] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:38:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:18.174132 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a22b2d-ed47-4927-9b6f-280f4591b0d5 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:18.181968 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:18.207503 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965cfd4b-eb49-4ea5-8f36-d264b471ecfd None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:18.247940 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa527a3-4c9a-49f1-b936-3ccbc5afd8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:18.248635 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa527a3-4c9a-49f1-b936-3ccbc5afd8a7 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:18.248913 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa527a3-4c9a-49f1-b936-3ccbc5afd8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:18.249202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa527a3-4c9a-49f1-b936-3ccbc5afd8a7 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:18.249924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa527a3-4c9a-49f1-b936-3ccbc5afd8a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:18.250284 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa527a3-4c9a-49f1-b936-3ccbc5afd8a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:18.251705 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa527a3-4c9a-49f1-b936-3ccbc5afd8a7 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:18.262065 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 372/755] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:38:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:18.268424 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa527a3-4c9a-49f1-b936-3ccbc5afd8a7 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:18.381924 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a22b2d-ed47-4927-9b6f-280f4591b0d5 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:18.385565 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:18.395749 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa527a3-4c9a-49f1-b936-3ccbc5afd8a7 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:18.427162 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05cd33-fb5b-4091-ac33-493f01f52294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:18.427924 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05cd33-fb5b-4091-ac33-493f01f52294 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:18.428208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05cd33-fb5b-4091-ac33-493f01f52294 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:18.428596 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05cd33-fb5b-4091-ac33-493f01f52294 None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:18.429684 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05cd33-fb5b-4091-ac33-493f01f52294 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:18.429998 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05cd33-fb5b-4091-ac33-493f01f52294 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:18.432829 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05cd33-fb5b-4091-ac33-493f01f52294 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:18.446218 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743650c1-c5b2-47b0-8b24-1813e9945bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:18.446912 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743650c1-c5b2-47b0-8b24-1813e9945bfc None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:18.454918 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743650c1-c5b2-47b0-8b24-1813e9945bfc None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:18.454918 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743650c1-c5b2-47b0-8b24-1813e9945bfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:18.456731 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 384/756] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:38:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:18.463226 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05cd33-fb5b-4091-ac33-493f01f52294 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:18.478837 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743650c1-c5b2-47b0-8b24-1813e9945bfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:18.480358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743650c1-c5b2-47b0-8b24-1813e9945bfc None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:18.518124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-743650c1-c5b2-47b0-8b24-1813e9945bfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:18.518124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:18.518124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:18.518124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:18.518124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:18.518124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:18.518124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:18.518124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:18.518124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:18.518124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:18.518124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:18.518124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:18.518124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:18.518124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:18.518124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:18.518124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:18.519282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:18.519282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:18.519282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:18.519282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:18.519282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:18.519282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:18.519282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:18.519282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:18.519282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:18.519282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:18.519282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:18.519282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:18.519282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:18.519282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:18.519282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:18.519282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:18.519282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:18.520261 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:18.520261 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:18.520261 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:18.520261 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 373/757] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:18.525542 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743650c1-c5b2-47b0-8b24-1813e9945bfc None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:18.569685 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:18.616952 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05cd33-fb5b-4091-ac33-493f01f52294 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:18.659250 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe135c99-e6dc-4fbc-9b0a-2153442297f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:18.659911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe135c99-e6dc-4fbc-9b0a-2153442297f3 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:18.660190 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe135c99-e6dc-4fbc-9b0a-2153442297f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:18.660506 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe135c99-e6dc-4fbc-9b0a-2153442297f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:18.661799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe135c99-e6dc-4fbc-9b0a-2153442297f3 None admin] RBAC: Adding query filter params (role.id=33ac296b51c143e7bb44211175752edb, scope.project.id=543d44293d894a7fbb83401f7a26dd9b, user.id=df07cffd82c74621855c8b8d7bfa989d) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:18.662110 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe135c99-e6dc-4fbc-9b0a-2153442297f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:18.663993 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe135c99-e6dc-4fbc-9b0a-2153442297f3 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:18.670839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 385/758] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:38:18 2022] GET /identity/v3/role_assignments?user.id=df07cffd82c74621855c8b8d7bfa989d&scope.project.id=543d44293d894a7fbb83401f7a26dd9b&role.id=33ac296b51c143e7bb44211175752edb => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:18.736775 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743650c1-c5b2-47b0-8b24-1813e9945bfc None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:18.780095 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1a33b-dbaa-459d-b0a8-1d60b8e21f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:18.780738 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1a33b-dbaa-459d-b0a8-1d60b8e21f4f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:18.781014 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1a33b-dbaa-459d-b0a8-1d60b8e21f4f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:18.781284 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1a33b-dbaa-459d-b0a8-1d60b8e21f4f None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:18.781950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1a33b-dbaa-459d-b0a8-1d60b8e21f4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:18.782251 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1a33b-dbaa-459d-b0a8-1d60b8e21f4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:18.783684 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1a33b-dbaa-459d-b0a8-1d60b8e21f4f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:18.802490 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 374/759] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:18.807972 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe135c99-e6dc-4fbc-9b0a-2153442297f3 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:18.936399 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe135c99-e6dc-4fbc-9b0a-2153442297f3 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:18.974754 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:18.985056 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca132633-3251-445e-8b7c-02086c384ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:18.985704 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca132633-3251-445e-8b7c-02086c384ff3 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:18.985981 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca132633-3251-445e-8b7c-02086c384ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:18.986249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca132633-3251-445e-8b7c-02086c384ff3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:18.995537 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca132633-3251-445e-8b7c-02086c384ff3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:18.996942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca132633-3251-445e-8b7c-02086c384ff3 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:19.009303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-ca132633-3251-445e-8b7c-02086c384ff3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 08:38:19.009303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:19.009303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:19.009303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:19.009303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:19.009303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:19.009303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:19.009303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:19.009303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:19.009303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:19.009303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:19.009303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:19.009303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:38:19.009303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:38:19.009303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:38:19.009303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:38:19.010402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:19.010402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:19.010402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:38:19.010402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:19.010402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:38:19.010402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:19.010402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:19.010402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:19.010402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:19.010402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:19.010402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:19.010402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:19.010402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:19.010402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:19.010402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:19.010402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:19.010402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:19.010402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:19.012562 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:19.012562 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:19.012562 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:38:19.012562 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:38:19.012562 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:38:19.012562 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:38:19.012562 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:38:19.012562 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:38:19.012562 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 08:38:19.012562 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:19.012562 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 386/760] 10.222.0.71 () {56 vars in 1093 bytes} [Fri Aug 26 08:38:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:19.020043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1a33b-dbaa-459d-b0a8-1d60b8e21f4f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:19.037566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:19.181016 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1a33b-dbaa-459d-b0a8-1d60b8e21f4f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:19.227817 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ddf369-260c-494e-806e-55f263412328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:19.228434 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ddf369-260c-494e-806e-55f263412328 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:19.228712 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ddf369-260c-494e-806e-55f263412328 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:19.228985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ddf369-260c-494e-806e-55f263412328 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:19.229747 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ddf369-260c-494e-806e-55f263412328 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:19.230075 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ddf369-260c-494e-806e-55f263412328 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:19.235641 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ddf369-260c-494e-806e-55f263412328 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:19.262324 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 375/761] 10.222.0.71 () {56 vars in 1133 bytes} [Fri Aug 26 08:38:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:19.267648 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca132633-3251-445e-8b7c-02086c384ff3 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:19.382191 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca132633-3251-445e-8b7c-02086c384ff3 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:19.418694 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15544f-3106-4a05-9f2e-50a826717539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:19.419372 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15544f-3106-4a05-9f2e-50a826717539 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:19.419651 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15544f-3106-4a05-9f2e-50a826717539 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:19.419921 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15544f-3106-4a05-9f2e-50a826717539 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:19.431519 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15544f-3106-4a05-9f2e-50a826717539 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:19.433463 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15544f-3106-4a05-9f2e-50a826717539 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:19.449521 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-da15544f-3106-4a05-9f2e-50a826717539 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:19.449521 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:19.449521 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:19.449521 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:19.449521 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:19.449521 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:19.449521 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:19.449521 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:19.449521 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:19.449521 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:19.449521 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:19.449521 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:19.449521 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:19.449521 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:19.449521 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:19.449521 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:19.450539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:19.450539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:19.450539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:19.450539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:19.450539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:19.450539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:19.450539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:19.450539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:19.450539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:19.450539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:19.450539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:19.450539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:19.450539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:19.450539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:19.450539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:19.450539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:19.450539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:19.451478 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:19.451478 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:19.451478 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:19.451478 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 387/762] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:19.459865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ddf369-260c-494e-806e-55f263412328 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:19.538984 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:19.599487 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ddf369-260c-494e-806e-55f263412328 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:19.615816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577ecac-1625-48a2-bc14-dc80879660dc None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:19.616144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577ecac-1625-48a2-bc14-dc80879660dc None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:19.616364 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577ecac-1625-48a2-bc14-dc80879660dc None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:19.618006 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 388/763] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:38:19.625171 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56c632-612a-44b9-af86-91d8c10f8dce None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:19.625632 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56c632-612a-44b9-af86-91d8c10f8dce None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:19.625910 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56c632-612a-44b9-af86-91d8c10f8dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:19.656611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eea50c-2e62-47e0-8e62-6555e8d0feb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:19.657233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eea50c-2e62-47e0-8e62-6555e8d0feb0 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:19.657511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eea50c-2e62-47e0-8e62-6555e8d0feb0 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:19.657828 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eea50c-2e62-47e0-8e62-6555e8d0feb0 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:19.658494 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eea50c-2e62-47e0-8e62-6555e8d0feb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:19.658843 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eea50c-2e62-47e0-8e62-6555e8d0feb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:19.660936 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eea50c-2e62-47e0-8e62-6555e8d0feb0 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:19.672083 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-1a56c632-612a-44b9-af86-91d8c10f8dce None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:19.673320 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1a56c632-612a-44b9-af86-91d8c10f8dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:19.674243 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 376/764] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:19.682287 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eea50c-2e62-47e0-8e62-6555e8d0feb0 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:19.735301 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:19.815427 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 389/765] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 08:38:19.867886 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eea50c-2e62-47e0-8e62-6555e8d0feb0 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:19.889894 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:19.906062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78b386-1687-40b7-b4a8-e76e3d3fab92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:19.906719 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78b386-1687-40b7-b4a8-e76e3d3fab92 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:19.907165 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78b386-1687-40b7-b4a8-e76e3d3fab92 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:19.907539 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78b386-1687-40b7-b4a8-e76e3d3fab92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:19.916460 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c78b386-1687-40b7-b4a8-e76e3d3fab92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:19.918035 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c78b386-1687-40b7-b4a8-e76e3d3fab92 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:19.931062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-5c78b386-1687-40b7-b4a8-e76e3d3fab92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:19.931062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:19.931062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:19.931062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:19.931062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:19.931062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:19.931062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:19.931062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:19.931062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:19.931062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:19.931062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:19.931062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:19.931062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:38:19.931062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:38:19.931062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:38:19.932180 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:38:19.932180 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:19.932180 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:19.932180 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:19.932180 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:19.932180 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:19.932180 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:19.932180 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:19.932180 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:19.932180 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:19.932180 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:19.932180 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:19.932180 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:19.932180 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:19.932180 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:19.932180 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:19.932180 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:19.933232 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:38:19.933232 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:38:19.933232 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:38:19.933232 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:38:19.933232 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:38:19.933232 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:38:19.933232 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:19.933232 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:19.933232 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 377/766] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:38:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:38:19.939673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a56c632-612a-44b9-af86-91d8c10f8dce None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:20.037188 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81646b2c-fc1d-4d28-be42-82f26c5aadb1 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:20.037811 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81646b2c-fc1d-4d28-be42-82f26c5aadb1 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:20.037811 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81646b2c-fc1d-4d28-be42-82f26c5aadb1 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:20.039429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 378/767] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:20.046087 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d6385-9d9c-4a6c-ac53-1d60fa3dbea1 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:20.046401 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d6385-9d9c-4a6c-ac53-1d60fa3dbea1 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:20.046632 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d6385-9d9c-4a6c-ac53-1d60fa3dbea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:20.075634 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-a40d6385-9d9c-4a6c-ac53-1d60fa3dbea1 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:20.076776 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a40d6385-9d9c-4a6c-ac53-1d60fa3dbea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:20.128594 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a56c632-612a-44b9-af86-91d8c10f8dce None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:20.164961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce679bfd-0f68-4842-bcc7-da982faf17ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:20.165584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce679bfd-0f68-4842-bcc7-da982faf17ea None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:20.165861 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce679bfd-0f68-4842-bcc7-da982faf17ea None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:20.166130 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce679bfd-0f68-4842-bcc7-da982faf17ea None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:20.166831 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce679bfd-0f68-4842-bcc7-da982faf17ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:20.167171 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce679bfd-0f68-4842-bcc7-da982faf17ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:20.168723 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce679bfd-0f68-4842-bcc7-da982faf17ea None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:20.181944 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 390/768] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:38:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:20.191300 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce679bfd-0f68-4842-bcc7-da982faf17ea None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:20.213968 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 379/769] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 08:38:20.243816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:20.305846 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d6385-9d9c-4a6c-ac53-1d60fa3dbea1 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:20.393930 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce679bfd-0f68-4842-bcc7-da982faf17ea None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:20.441297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d6385-9d9c-4a6c-ac53-1d60fa3dbea1 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:20.459400 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:20.479758 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761a63e-6a8d-4131-a1a2-039ba61a75f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:20.480481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761a63e-6a8d-4131-a1a2-039ba61a75f7 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:20.480765 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761a63e-6a8d-4131-a1a2-039ba61a75f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:20.481095 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761a63e-6a8d-4131-a1a2-039ba61a75f7 None admin] PATH_INFO: `/v3/services/network` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:20.481692 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761a63e-6a8d-4131-a1a2-039ba61a75f7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:20.483152 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761a63e-6a8d-4131-a1a2-039ba61a75f7 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:20.492812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-5761a63e-6a8d-4131-a1a2-039ba61a75f7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 08:38:20.492812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:20.492812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 08:38:20.492812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 08:38:20.492812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 08:38:20.492812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 08:38:20.492812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 08:38:20.492812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:20.492812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 08:38:20.492812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:20.492812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:38:20.492812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:20.492812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:20.492812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:20.492812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:20.492812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:20.492812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:20.494255 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:20.494255 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:20.494255 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:20.494255 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:20.494255 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:20.494255 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:20.494255 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 08:38:20.494255 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 08:38:20.494255 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 08:38:20.494255 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 08:38:20.494255 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:20.494255 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:20.494255 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:20.494255 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:20.494255 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:20.494255 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:20.495173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:20.495173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:20.495173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:20.495173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:20.495173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:20.495173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:20.495173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:20.495173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:20.495173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:20.495173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:20.495173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 08:38:20.495173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:20.495173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 08:38:20.495173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:20.495173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 380/770] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:38:20 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:20.500453 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761a63e-6a8d-4131-a1a2-039ba61a75f7 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:20.522256 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3420fec5-fc20-4453-802e-1b68aeb4da32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:20.523240 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420fec5-fc20-4453-802e-1b68aeb4da32 None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:20.523240 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420fec5-fc20-4453-802e-1b68aeb4da32 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:20.523496 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420fec5-fc20-4453-802e-1b68aeb4da32 None admin] PATH_INFO: `/v3/projects/543d44293d894a7fbb83401f7a26dd9b/users/f16d1768715a41648af3fe6afadd7477/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:20.525976 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3420fec5-fc20-4453-802e-1b68aeb4da32 None admin] RBAC: Authorizing `identity:create_grant(project_id=543d44293d894a7fbb83401f7a26dd9b, user_id=f16d1768715a41648af3fe6afadd7477, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:20.527570 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3420fec5-fc20-4453-802e-1b68aeb4da32 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:20.551308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 391/771] 10.222.0.71 () {58 vars in 1439 bytes} [Fri Aug 26 08:38:20 2022] PUT /identity/v3/projects/543d44293d894a7fbb83401f7a26dd9b/users/f16d1768715a41648af3fe6afadd7477/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:38:20.552364 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3420fec5-fc20-4453-802e-1b68aeb4da32 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:20.641140 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761a63e-6a8d-4131-a1a2-039ba61a75f7 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:20.682549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bd21b-b74b-4d77-a3b8-7624db851615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:20.683533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bd21b-b74b-4d77-a3b8-7624db851615 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:20.683977 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bd21b-b74b-4d77-a3b8-7624db851615 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:20.684398 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bd21b-b74b-4d77-a3b8-7624db851615 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:20.685325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8bd21b-b74b-4d77-a3b8-7624db851615 None admin] RBAC: Adding query filter params (name=network) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:20.685896 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8bd21b-b74b-4d77-a3b8-7624db851615 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:20.687601 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8bd21b-b74b-4d77-a3b8-7624db851615 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:20.693767 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 381/772] 10.222.0.71 () {56 vars in 1112 bytes} [Fri Aug 26 08:38:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:20.699878 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bd21b-b74b-4d77-a3b8-7624db851615 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:20.752503 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:20.769213 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:20.786847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3420fec5-fc20-4453-802e-1b68aeb4da32 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:20.827116 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bd21b-b74b-4d77-a3b8-7624db851615 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:20.837043 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab29de-9fdd-4d96-801d-37ac3bfe1a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:20.839450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab29de-9fdd-4d96-801d-37ac3bfe1a7e None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:20.840205 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab29de-9fdd-4d96-801d-37ac3bfe1a7e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:20.841238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab29de-9fdd-4d96-801d-37ac3bfe1a7e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:20.842390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab29de-9fdd-4d96-801d-37ac3bfe1a7e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:20.844685 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab29de-9fdd-4d96-801d-37ac3bfe1a7e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:20.860487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-6dab29de-9fdd-4d96-801d-37ac3bfe1a7e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 08:38:20.860487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:20.860487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 08:38:20.860487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 08:38:20.860487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 08:38:20.860487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 08:38:20.860487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 08:38:20.860487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:20.860487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 08:38:20.860487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:20.860487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:38:20.860487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:20.860487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:20.860487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:20.860487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:20.860487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:20.860487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:20.861776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:20.861776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:20.861776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:20.861776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:20.861776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:20.861776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:20.861776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 08:38:20.861776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 08:38:20.861776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 08:38:20.861776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 08:38:20.861776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:20.861776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:20.861776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:20.861776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:20.861776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:20.861776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:20.862793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:20.862793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:20.862793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:20.862793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:20.862793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:20.862793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:20.862793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:20.862793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:20.862793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:20.862793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:20.862793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 08:38:20.862793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:20.862793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 08:38:20.862793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:20.866337 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 392/773] 10.222.0.71 () {56 vars in 1129 bytes} [Fri Aug 26 08:38:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:20.868270 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76655f3-b989-478d-8268-1617d7cd1b6a None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:20.869111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76655f3-b989-478d-8268-1617d7cd1b6a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:20.869587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76655f3-b989-478d-8268-1617d7cd1b6a None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:20.869802 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce11c7-f27e-4175-80a5-50b39d93bc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:20.870520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce11c7-f27e-4175-80a5-50b39d93bc19 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:20.870899 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce11c7-f27e-4175-80a5-50b39d93bc19 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:20.871439 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce11c7-f27e-4175-80a5-50b39d93bc19 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:20.872069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce11c7-f27e-4175-80a5-50b39d93bc19 None admin] RBAC: Adding query filter params (type=network) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:20.872408 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 393/774] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:20.872521 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce11c7-f27e-4175-80a5-50b39d93bc19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:20.874944 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76655f3-b989-478d-8268-1617d7cd1b6a None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:20.875118 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce11c7-f27e-4175-80a5-50b39d93bc19 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:20.882273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 382/775] 10.222.0.71 () {56 vars in 1112 bytes} [Fri Aug 26 08:38:20 2022] GET /identity/v3/services?type=network => generated 350 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:20.883749 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101ba8a-fa23-40e7-9938-53a7feb52f1a None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:20.884123 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101ba8a-fa23-40e7-9938-53a7feb52f1a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:20.884487 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101ba8a-fa23-40e7-9938-53a7feb52f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:20.915774 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c101ba8a-fa23-40e7-9938-53a7feb52f1a None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:20.917156 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c101ba8a-fa23-40e7-9938-53a7feb52f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:21.005336 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76655f3-b989-478d-8268-1617d7cd1b6a None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:21.014982 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 383/776] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:21.015931 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c101ba8a-fa23-40e7-9938-53a7feb52f1a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:21.064683 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef71ab-5293-478b-bb4c-e8749d38fa30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:21.065292 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef71ab-5293-478b-bb4c-e8749d38fa30 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:21.065566 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef71ab-5293-478b-bb4c-e8749d38fa30 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:21.065833 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef71ab-5293-478b-bb4c-e8749d38fa30 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:21.067387 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef71ab-5293-478b-bb4c-e8749d38fa30 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:21.067687 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef71ab-5293-478b-bb4c-e8749d38fa30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:21.070314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef71ab-5293-478b-bb4c-e8749d38fa30 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:21.076826 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:21.080254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 394/777] 10.222.0.71 () {56 vars in 1124 bytes} [Fri Aug 26 08:38:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:21.084259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac55f58-bb86-4046-9239-3b64c8bcb086 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:21.084544 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac55f58-bb86-4046-9239-3b64c8bcb086 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:21.084807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac55f58-bb86-4046-9239-3b64c8bcb086 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:21.086099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 395/778] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:21.086779 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac55f58-bb86-4046-9239-3b64c8bcb086 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:21.113239 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:21.164314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c101ba8a-fa23-40e7-9938-53a7feb52f1a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:21.201498 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c688e-f435-4127-8cd4-baa231720770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:21.202286 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c688e-f435-4127-8cd4-baa231720770 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:21.202662 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c688e-f435-4127-8cd4-baa231720770 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:21.203064 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c688e-f435-4127-8cd4-baa231720770 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:21.203968 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c688e-f435-4127-8cd4-baa231720770 None admin] RBAC: Adding query filter params (interface=public, service_id=c1a9a6cd5078435b8ea6a6559fba335b, region_id=RegionOne) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:21.204386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c688e-f435-4127-8cd4-baa231720770 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:21.206070 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c688e-f435-4127-8cd4-baa231720770 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:21.207766 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac55f58-bb86-4046-9239-3b64c8bcb086 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:21.219386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 384/779] 10.222.0.71 () {56 vars in 1251 bytes} [Fri Aug 26 08:38:21 2022] GET /identity/v3/endpoints?service_id=c1a9a6cd5078435b8ea6a6559fba335b&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:21.219386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2c3ef-86b2-493b-aa69-62cc2127c473 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:21.219386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2c3ef-86b2-493b-aa69-62cc2127c473 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:21.219386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2c3ef-86b2-493b-aa69-62cc2127c473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:21.238819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c7f2c3ef-86b2-493b-aa69-62cc2127c473 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:21.240174 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c7f2c3ef-86b2-493b-aa69-62cc2127c473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:21.251497 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1147de-58bc-44ed-befb-4b4815110185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:21.252498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1147de-58bc-44ed-befb-4b4815110185 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:21.252942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1147de-58bc-44ed-befb-4b4815110185 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:21.253443 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1147de-58bc-44ed-befb-4b4815110185 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:21.254248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1147de-58bc-44ed-befb-4b4815110185 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:21.254873 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1147de-58bc-44ed-befb-4b4815110185 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:21.256854 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1147de-58bc-44ed-befb-4b4815110185 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:21.268611 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 396/780] 10.222.0.71 () {56 vars in 1124 bytes} [Fri Aug 26 08:38:21 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:21.269835 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1147de-58bc-44ed-befb-4b4815110185 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:21.317026 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 385/781] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:21.317912 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2c3ef-86b2-493b-aa69-62cc2127c473 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:21.345800 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:21.367293 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:21.387115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1147de-58bc-44ed-befb-4b4815110185 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:21.421817 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2c3ef-86b2-493b-aa69-62cc2127c473 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:21.424290 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af822b2d-7e9a-410d-94fe-9f0a9422b5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:21.425134 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af822b2d-7e9a-410d-94fe-9f0a9422b5c1 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:21.425611 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af822b2d-7e9a-410d-94fe-9f0a9422b5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:21.426081 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af822b2d-7e9a-410d-94fe-9f0a9422b5c1 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:21.426875 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af822b2d-7e9a-410d-94fe-9f0a9422b5c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:21.428552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af822b2d-7e9a-410d-94fe-9f0a9422b5c1 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:21.434634 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 397/782] 10.222.0.71 () {56 vars in 1087 bytes} [Fri Aug 26 08:38:21 2022] GET /identity/v3/services => generated 772 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:21.435909 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af822b2d-7e9a-410d-94fe-9f0a9422b5c1 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:21.459609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1314c-be69-46ad-afa7-733ce0087a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:21.460135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1314c-be69-46ad-afa7-733ce0087a7c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:21.460413 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1314c-be69-46ad-afa7-733ce0087a7c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:21.460684 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1314c-be69-46ad-afa7-733ce0087a7c None admin] PATH_INFO: `/v3/endpoints` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:21.461395 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d1314c-be69-46ad-afa7-733ce0087a7c None admin] RBAC: Adding query filter params (interface=public, service_id=adc3cb2b4bb94f039b910a8eb7e68532, region_id=RegionOne) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:21.461702 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d1314c-be69-46ad-afa7-733ce0087a7c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:21.463549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d1314c-be69-46ad-afa7-733ce0087a7c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:21.470819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 386/783] 10.222.0.71 () {56 vars in 1251 bytes} [Fri Aug 26 08:38:21 2022] GET /identity/v3/endpoints?service_id=adc3cb2b4bb94f039b910a8eb7e68532&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:21.475759 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1314c-be69-46ad-afa7-733ce0087a7c None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:21.622563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af822b2d-7e9a-410d-94fe-9f0a9422b5c1 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:21.659186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:21.668013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98715710-91cf-4694-849e-0bef1b6de1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:21.669246 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98715710-91cf-4694-849e-0bef1b6de1b1 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:21.669246 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98715710-91cf-4694-849e-0bef1b6de1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:21.669246 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98715710-91cf-4694-849e-0bef1b6de1b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:21.675315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1314c-be69-46ad-afa7-733ce0087a7c None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:21.686803 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98715710-91cf-4694-849e-0bef1b6de1b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:38:21.693323 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-98715710-91cf-4694-849e-0bef1b6de1b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:38:21.693323 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:21.693323 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:21.693323 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:21.693323 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:21.693323 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:21.693323 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:21.693323 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:21.693323 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:21.693323 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:21.693323 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:21.693323 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:21.693323 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:38:21.693323 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:38:21.693323 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:38:21.693323 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:38:21.694297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:38:21.694297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:38:21.694297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:38:21.694297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:38:21.694297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:21.694297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:21.694297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:21.694297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:21.694297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:21.694297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:21.694297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:21.694297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:21.694297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:21.694297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:21.694297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:21.694297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:21.695166 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:21.695166 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:21.695166 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:21.695166 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:21.695166 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:38:21.695166 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:38:21.695166 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:38:21.695166 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:38:21.695166 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:38:21.695166 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:38:21.695166 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:38:21.695166 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:21.698403 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 398/784] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:38:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:21.699571 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98715710-91cf-4694-849e-0bef1b6de1b1 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:21.704355 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:21.722543 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d18991-a3ab-4b84-8909-1ffb64b43512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:21.723123 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d18991-a3ab-4b84-8909-1ffb64b43512 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:21.723403 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d18991-a3ab-4b84-8909-1ffb64b43512 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:21.723741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d18991-a3ab-4b84-8909-1ffb64b43512 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:21.724354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d18991-a3ab-4b84-8909-1ffb64b43512 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:21.725835 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d18991-a3ab-4b84-8909-1ffb64b43512 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:21.731518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 387/785] 10.222.0.71 () {56 vars in 1087 bytes} [Fri Aug 26 08:38:21 2022] GET /identity/v3/services => generated 772 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:21.732396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d18991-a3ab-4b84-8909-1ffb64b43512 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:21.888659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98715710-91cf-4694-849e-0bef1b6de1b1 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:21.910276 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d18991-a3ab-4b84-8909-1ffb64b43512 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:21.932334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec2c0d-533a-4a64-919b-b7aad3b4bf8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:21.933445 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec2c0d-533a-4a64-919b-b7aad3b4bf8c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:21.933945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec2c0d-533a-4a64-919b-b7aad3b4bf8c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:21.934446 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec2c0d-533a-4a64-919b-b7aad3b4bf8c None admin] PATH_INFO: `/v3/roles/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:21.949264 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ec2c0d-533a-4a64-919b-b7aad3b4bf8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:38:21.954643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-71ec2c0d-533a-4a64-919b-b7aad3b4bf8c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:38:21.954643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:21.954643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:21.954643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:21.954643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:21.954643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:21.954643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:21.954643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:21.954643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:21.954643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:21.954643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:21.954643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:21.954643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:38:21.954643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:38:21.954643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:38:21.954643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:38:21.955725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:38:21.955725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:38:21.955725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:38:21.955725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:38:21.955725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:21.955725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:21.955725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:21.955725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:21.955725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:21.955725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:21.955725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:21.955725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:21.955725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:21.955725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:21.955725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:21.955725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:21.956568 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:21.956568 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:21.956568 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:21.956568 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:21.956568 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:38:21.956568 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:38:21.956568 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:38:21.956568 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:38:21.956568 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:38:21.956568 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:38:21.956568 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:38:21.956568 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:21.962934 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 399/786] 10.222.0.71 () {56 vars in 1102 bytes} [Fri Aug 26 08:38:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:21.966068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec2c0d-533a-4a64-919b-b7aad3b4bf8c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:22.043063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:22.107003 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec2c0d-533a-4a64-919b-b7aad3b4bf8c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:22.132984 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb062ea8-a346-4cd4-8da7-05fec457a489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:22.133708 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb062ea8-a346-4cd4-8da7-05fec457a489 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:22.133874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb062ea8-a346-4cd4-8da7-05fec457a489 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:22.134140 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb062ea8-a346-4cd4-8da7-05fec457a489 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:22.134816 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb062ea8-a346-4cd4-8da7-05fec457a489 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:22.135147 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb062ea8-a346-4cd4-8da7-05fec457a489 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:22.139598 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb062ea8-a346-4cd4-8da7-05fec457a489 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:22.160666 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 388/787] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:38:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:22.163772 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb062ea8-a346-4cd4-8da7-05fec457a489 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:22.186656 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2ac84-493a-44b2-8545-1aa3a20939e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:22.187301 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2ac84-493a-44b2-8545-1aa3a20939e1 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:22.187576 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2ac84-493a-44b2-8545-1aa3a20939e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:22.187846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2ac84-493a-44b2-8545-1aa3a20939e1 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:22.188522 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2ac84-493a-44b2-8545-1aa3a20939e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:22.188825 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2ac84-493a-44b2-8545-1aa3a20939e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:22.190208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2ac84-493a-44b2-8545-1aa3a20939e1 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:22.205373 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:22.219752 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 400/788] 10.222.0.71 () {56 vars in 1103 bytes} [Fri Aug 26 08:38:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:22.236725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2ac84-493a-44b2-8545-1aa3a20939e1 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:22.351409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2ac84-493a-44b2-8545-1aa3a20939e1 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:22.384742 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:22.395634 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1e02a-cced-4ac4-b3fc-e37b7c5b5a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:22.396310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1e02a-cced-4ac4-b3fc-e37b7c5b5a8b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:22.396588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1e02a-cced-4ac4-b3fc-e37b7c5b5a8b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:22.396856 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1e02a-cced-4ac4-b3fc-e37b7c5b5a8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:22.419431 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1e02a-cced-4ac4-b3fc-e37b7c5b5a8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:22.423044 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1e02a-cced-4ac4-b3fc-e37b7c5b5a8b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:22.438982 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb062ea8-a346-4cd4-8da7-05fec457a489 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:22.450844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-74f1e02a-cced-4ac4-b3fc-e37b7c5b5a8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:22.450844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:22.450844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:22.450844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:22.450844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:22.450844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:22.450844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:22.450844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:22.450844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:22.450844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:22.450844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:22.450844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:22.450844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:22.450844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:22.450844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:22.450844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:22.451966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:22.451966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:22.451966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:22.451966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:22.451966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:22.451966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:22.451966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:22.451966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:22.451966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:22.451966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:22.451966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:22.451966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:22.451966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:22.451966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:22.451966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:22.451966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:22.451966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:22.452875 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:22.452875 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:22.452875 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:22.466317 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 401/789] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:38:22.477608 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1e02a-cced-4ac4-b3fc-e37b7c5b5a8b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:22.527244 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bded3b7-547c-4438-9b48-0ed7d41fe9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:22.527958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bded3b7-547c-4438-9b48-0ed7d41fe9ce None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:22.528235 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bded3b7-547c-4438-9b48-0ed7d41fe9ce None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:22.528502 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bded3b7-547c-4438-9b48-0ed7d41fe9ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:22.556095 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bded3b7-547c-4438-9b48-0ed7d41fe9ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:22.557606 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bded3b7-547c-4438-9b48-0ed7d41fe9ce None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:22.590359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-5bded3b7-547c-4438-9b48-0ed7d41fe9ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:22.590359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:22.590359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:22.590359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:22.590359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:22.590359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:22.590359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:22.590359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:22.590359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:22.590359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:22.590359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:22.590359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:22.590359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:22.590359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:22.590359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:22.590359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:22.592718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:22.592718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:22.592718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:22.592718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:22.592718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:22.592718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:22.592718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:22.592718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:22.592718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:22.592718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:22.592718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:22.592718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:22.592718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:22.592718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:22.592718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:22.592718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:22.592718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:22.594813 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:22.594813 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:22.594813 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:22.596144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 389/790] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 433 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:22.601326 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bded3b7-547c-4438-9b48-0ed7d41fe9ce None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:22.670712 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1e02a-cced-4ac4-b3fc-e37b7c5b5a8b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:22.703073 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:22.713161 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4ee35-fc69-4e50-8635-6e1482088b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:22.713920 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4ee35-fc69-4e50-8635-6e1482088b2f None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:22.714266 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4ee35-fc69-4e50-8635-6e1482088b2f None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:22.714603 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4ee35-fc69-4e50-8635-6e1482088b2f None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:22.715473 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4ee35-fc69-4e50-8635-6e1482088b2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:22.715848 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4ee35-fc69-4e50-8635-6e1482088b2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:22.717950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb4ee35-fc69-4e50-8635-6e1482088b2f None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:22.736694 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 402/791] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:22.738352 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8d5ee-d9d2-411e-af3d-2663db22860f None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:22.738996 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8d5ee-d9d2-411e-af3d-2663db22860f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:22.739333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8d5ee-d9d2-411e-af3d-2663db22860f None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:22.746461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 403/792] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:22.746461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8d5ee-d9d2-411e-af3d-2663db22860f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:22.754505 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bded3b7-547c-4438-9b48-0ed7d41fe9ce None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:22.798363 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:22.799233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fdae2-772e-40f3-8697-d52191e8274f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:22.799696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fdae2-772e-40f3-8697-d52191e8274f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:22.801789 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fdae2-772e-40f3-8697-d52191e8274f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:22.801789 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fdae2-772e-40f3-8697-d52191e8274f None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:22.802015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8fdae2-772e-40f3-8697-d52191e8274f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:22.802465 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8fdae2-772e-40f3-8697-d52191e8274f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:22.804429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8fdae2-772e-40f3-8697-d52191e8274f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:22.824878 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 390/793] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:22.826164 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103bc31-0e66-4b25-9c7f-a53379007825 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:22.826443 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103bc31-0e66-4b25-9c7f-a53379007825 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:22.826711 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103bc31-0e66-4b25-9c7f-a53379007825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:22.852089 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-9103bc31-0e66-4b25-9c7f-a53379007825 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:22.853944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9103bc31-0e66-4b25-9c7f-a53379007825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:22.882734 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8d5ee-d9d2-411e-af3d-2663db22860f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:22.931419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d5f43-60d7-46fa-9387-9d88d4efee8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:22.932158 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d5f43-60d7-46fa-9387-9d88d4efee8c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:22.932565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d5f43-60d7-46fa-9387-9d88d4efee8c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:22.933025 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d5f43-60d7-46fa-9387-9d88d4efee8c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:22.941942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d5f43-60d7-46fa-9387-9d88d4efee8c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:22.943784 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d5f43-60d7-46fa-9387-9d88d4efee8c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:22.962950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-055d5f43-60d7-46fa-9387-9d88d4efee8c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 08:38:22.962950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:22.962950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:22.962950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:22.962950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:22.962950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:22.962950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:22.962950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:22.962950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:22.962950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:22.962950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:22.962950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:22.962950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:38:22.962950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:38:22.962950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:38:22.962950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:38:22.964023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:22.964023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:22.964023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:38:22.964023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:22.964023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:38:22.964023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:22.964023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:22.964023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:22.964023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:22.964023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:22.964023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:22.964023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:22.964023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:22.964023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:22.964023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:22.964023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:22.964023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:22.964023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:22.964967 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:22.964967 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:22.964967 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:38:22.964967 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:38:22.964967 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:38:22.964967 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:38:22.964967 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:38:22.964967 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:38:22.964967 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 08:38:22.964967 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:22.964967 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 404/794] 10.222.0.71 () {56 vars in 1117 bytes} [Fri Aug 26 08:38:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:22.964967 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d5f43-60d7-46fa-9387-9d88d4efee8c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:22.966313 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 391/795] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:22.967489 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103bc31-0e66-4b25-9c7f-a53379007825 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:23.042742 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:23.125799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:23.129470 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d5f43-60d7-46fa-9387-9d88d4efee8c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:23.136432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103bc31-0e66-4b25-9c7f-a53379007825 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:23.181224 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3fc98c-0341-4473-8b86-91ebd087b3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:23.182302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fc98c-0341-4473-8b86-91ebd087b3e4 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:23.182781 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fc98c-0341-4473-8b86-91ebd087b3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:23.183285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fc98c-0341-4473-8b86-91ebd087b3e4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:23.205452 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff66de-c0e8-40f4-96c9-8063d5142b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:23.206525 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff66de-c0e8-40f4-96c9-8063d5142b6c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:23.207255 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff66de-c0e8-40f4-96c9-8063d5142b6c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:23.207855 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff66de-c0e8-40f4-96c9-8063d5142b6c None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:23.208224 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3fc98c-0341-4473-8b86-91ebd087b3e4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:23.209121 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff66de-c0e8-40f4-96c9-8063d5142b6c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:23.209634 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff66de-c0e8-40f4-96c9-8063d5142b6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:23.209758 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3fc98c-0341-4473-8b86-91ebd087b3e4 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:23.211416 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff66de-c0e8-40f4-96c9-8063d5142b6c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:23.232896 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 392/796] 10.222.0.71 () {56 vars in 1149 bytes} [Fri Aug 26 08:38:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:23.241347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-7a3fc98c-0341-4473-8b86-91ebd087b3e4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 08:38:23.241347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:23.241347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:23.241347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:23.241347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:23.241347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:23.241347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:23.241347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:23.241347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:23.241347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:23.241347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:23.241347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:23.241347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:38:23.241347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:38:23.241347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:38:23.241347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:38:23.242368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:23.242368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:23.242368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:38:23.242368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:23.242368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:38:23.242368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:23.242368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:23.242368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:23.242368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:23.242368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:23.242368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:23.242368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:23.242368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:23.242368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:23.242368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:23.242368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:23.242368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:23.242368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:23.245591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:23.245591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:23.245591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:38:23.245591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:38:23.245591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:38:23.245591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:38:23.245591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:38:23.245591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:38:23.245591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 08:38:23.245591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:23.247463 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff66de-c0e8-40f4-96c9-8063d5142b6c None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:23.259943 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 405/797] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:38:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:23.266081 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3fc98c-0341-4473-8b86-91ebd087b3e4 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:23.405314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3fc98c-0341-4473-8b86-91ebd087b3e4 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:23.422124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff66de-c0e8-40f4-96c9-8063d5142b6c None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:23.448000 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:23.453183 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63728f7a-2183-472b-ad16-557f4af0416d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:23.454397 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63728f7a-2183-472b-ad16-557f4af0416d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:23.454822 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63728f7a-2183-472b-ad16-557f4af0416d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:23.455580 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63728f7a-2183-472b-ad16-557f4af0416d None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:23.456920 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63728f7a-2183-472b-ad16-557f4af0416d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:23.457624 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63728f7a-2183-472b-ad16-557f4af0416d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:23.460054 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63728f7a-2183-472b-ad16-557f4af0416d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:23.465086 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f745143-a643-4a94-b502-bcae58196064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:23.465907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f745143-a643-4a94-b502-bcae58196064 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:23.466275 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f745143-a643-4a94-b502-bcae58196064 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:23.466643 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f745143-a643-4a94-b502-bcae58196064 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:23.482615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f745143-a643-4a94-b502-bcae58196064 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:23.484113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 406/798] 10.222.0.71 () {56 vars in 1135 bytes} [Fri Aug 26 08:38:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:23.484562 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f745143-a643-4a94-b502-bcae58196064 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:23.488866 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63728f7a-2183-472b-ad16-557f4af0416d None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:23.508751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:23.509254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-4f745143-a643-4a94-b502-bcae58196064 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:23.509254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:23.509254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:23.509254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:23.509254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:23.509254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:23.509254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:23.509254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:23.509254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:23.509254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:23.509254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:23.509254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:23.509254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:23.509254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:23.509254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:23.509254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:23.510175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:23.510175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:23.510175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:23.510175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:23.510175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:23.510175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:23.510175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:23.510175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:23.510175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:23.510175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:23.510175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:23.510175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:23.510175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:23.510175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:23.510175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:23.510175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:23.510175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:23.511178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:23.511178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:23.511178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:23.517789 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 393/799] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:23.517789 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f745143-a643-4a94-b502-bcae58196064 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:23.620487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63728f7a-2183-472b-ad16-557f4af0416d None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:23.657246 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f745143-a643-4a94-b502-bcae58196064 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:23.668190 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca11a2b-7ecd-4a9b-9791-a4267fcd0421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:23.669695 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca11a2b-7ecd-4a9b-9791-a4267fcd0421 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:23.669903 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca11a2b-7ecd-4a9b-9791-a4267fcd0421 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:23.670188 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca11a2b-7ecd-4a9b-9791-a4267fcd0421 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:23.687651 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca11a2b-7ecd-4a9b-9791-a4267fcd0421 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:38:23.692899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-4ca11a2b-7ecd-4a9b-9791-a4267fcd0421 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:38:23.692899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:23.692899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:23.692899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:23.692899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:23.692899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:23.692899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:23.692899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:23.692899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:23.692899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:23.692899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:23.692899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:23.692899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:38:23.692899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:38:23.692899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:38:23.692899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:38:23.694212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:38:23.694212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:38:23.694212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:38:23.694212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:38:23.694212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:23.694212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:23.694212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:23.694212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:23.694212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:23.694212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:23.694212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:23.694212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:23.694212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:23.694212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:23.694212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:23.694212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:23.695673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:23.695673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:23.695673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:23.695673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:23.695673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:38:23.695673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:38:23.695673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:38:23.695673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:38:23.695673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:38:23.695673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:38:23.695673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:38:23.695673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:23.695673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 407/800] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:38:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:23.695673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca11a2b-7ecd-4a9b-9791-a4267fcd0421 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:23.706968 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7185fef-1a88-4324-92b9-4e5aa7adbeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:23.707511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7185fef-1a88-4324-92b9-4e5aa7adbeba None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:23.707792 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7185fef-1a88-4324-92b9-4e5aa7adbeba None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:23.708061 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7185fef-1a88-4324-92b9-4e5aa7adbeba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:23.722264 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7185fef-1a88-4324-92b9-4e5aa7adbeba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:23.724197 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7185fef-1a88-4324-92b9-4e5aa7adbeba None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:23.759001 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-d7185fef-1a88-4324-92b9-4e5aa7adbeba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:23.759001 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:23.759001 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:23.759001 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:23.759001 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:23.759001 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:23.759001 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:23.759001 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:23.759001 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:23.759001 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:23.759001 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:23.759001 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:23.759001 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:23.759001 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:23.759001 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:23.759001 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:23.760645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:23.760645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:23.760645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:23.760645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:23.760645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:23.760645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:23.760645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:23.760645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:23.760645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:23.760645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:23.760645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:23.760645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:23.760645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:23.760645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:23.760645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:23.760645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:23.760645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:23.763615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:23.763615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:23.763615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:23.763615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 394/801] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:23.765635 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bd6ef-4a58-42f6-8b7a-e6653cc6cf3f None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:23.766167 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bd6ef-4a58-42f6-8b7a-e6653cc6cf3f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:23.766726 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bd6ef-4a58-42f6-8b7a-e6653cc6cf3f None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:23.775803 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 395/802] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:23.775803 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bd6ef-4a58-42f6-8b7a-e6653cc6cf3f None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:23.808017 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:23.843571 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:23.887480 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca11a2b-7ecd-4a9b-9791-a4267fcd0421 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:23.941100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a7661-1fc1-4831-8fa0-03a11cc85b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:23.941996 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a7661-1fc1-4831-8fa0-03a11cc85b57 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:23.941996 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a7661-1fc1-4831-8fa0-03a11cc85b57 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:23.942337 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a7661-1fc1-4831-8fa0-03a11cc85b57 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:23.943047 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53a7661-1fc1-4831-8fa0-03a11cc85b57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:23.943354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53a7661-1fc1-4831-8fa0-03a11cc85b57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:23.945472 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53a7661-1fc1-4831-8fa0-03a11cc85b57 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:23.966956 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bd6ef-4a58-42f6-8b7a-e6653cc6cf3f None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:23.974976 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 408/803] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:23.974976 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a7661-1fc1-4831-8fa0-03a11cc85b57 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:24.018947 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de4693-a93f-4249-a68c-b81ead078d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:24.019846 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de4693-a93f-4249-a68c-b81ead078d7c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:24.020118 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de4693-a93f-4249-a68c-b81ead078d7c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:24.020382 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de4693-a93f-4249-a68c-b81ead078d7c None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:24.021098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de4693-a93f-4249-a68c-b81ead078d7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:24.021616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de4693-a93f-4249-a68c-b81ead078d7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:24.023093 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de4693-a93f-4249-a68c-b81ead078d7c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:24.033884 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 396/804] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:38:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:24.035215 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1687f-d831-4f6d-96a7-4cdfb8363190 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:24.035505 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1687f-d831-4f6d-96a7-4cdfb8363190 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:24.035758 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1687f-d831-4f6d-96a7-4cdfb8363190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:24.057871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-f8d1687f-d831-4f6d-96a7-4cdfb8363190 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:24.059263 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f8d1687f-d831-4f6d-96a7-4cdfb8363190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:24.111229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a7661-1fc1-4831-8fa0-03a11cc85b57 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:24.138914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 397/805] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:24.139793 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1687f-d831-4f6d-96a7-4cdfb8363190 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:24.158197 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957dbbe8-0439-4eee-9bd4-496730fb3956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:24.159351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957dbbe8-0439-4eee-9bd4-496730fb3956 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:24.159785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957dbbe8-0439-4eee-9bd4-496730fb3956 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:24.160224 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957dbbe8-0439-4eee-9bd4-496730fb3956 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:24.161265 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957dbbe8-0439-4eee-9bd4-496730fb3956 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:24.161689 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:24.161807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957dbbe8-0439-4eee-9bd4-496730fb3956 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:24.164290 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957dbbe8-0439-4eee-9bd4-496730fb3956 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:24.169210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:24.189483 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 409/806] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:24.192370 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789b088-a4b8-4bce-be7f-655744241a72 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:24.192370 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789b088-a4b8-4bce-be7f-655744241a72 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:24.192370 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789b088-a4b8-4bce-be7f-655744241a72 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:24.193893 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 410/807] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:24.194360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789b088-a4b8-4bce-be7f-655744241a72 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:24.301748 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1687f-d831-4f6d-96a7-4cdfb8363190 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:24.324676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789b088-a4b8-4bce-be7f-655744241a72 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:24.353041 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f68d5f-36ea-48b0-982b-1d23a3eae92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:24.354314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f68d5f-36ea-48b0-982b-1d23a3eae92e None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:24.354863 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f68d5f-36ea-48b0-982b-1d23a3eae92e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:24.355442 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f68d5f-36ea-48b0-982b-1d23a3eae92e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:24.365149 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f68d5f-36ea-48b0-982b-1d23a3eae92e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:24.366981 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f68d5f-36ea-48b0-982b-1d23a3eae92e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:24.384049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-25f68d5f-36ea-48b0-982b-1d23a3eae92e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:24.384049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:24.384049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:24.384049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:24.384049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:24.384049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:24.384049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:24.384049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:24.384049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:24.384049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:24.384049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:24.384049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:24.384049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:38:24.384049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:38:24.384049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:38:24.385025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:38:24.385025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:24.385025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:24.385025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:24.385025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:24.385025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:24.385025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:24.385025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:24.385025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:24.385025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:24.385025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:24.385025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:24.385025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:24.385025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:24.385025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:24.385025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:24.385025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:24.385913 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:38:24.385913 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:38:24.385913 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:38:24.385913 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:38:24.385913 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:38:24.385913 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:38:24.385913 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:24.385913 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:24.385913 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 398/808] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:38:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:24.386422 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f68d5f-36ea-48b0-982b-1d23a3eae92e None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:24.401654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0dd52c-0948-441b-868d-e9d6f62d5a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:24.402681 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0dd52c-0948-441b-868d-e9d6f62d5a0c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:24.403249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0dd52c-0948-441b-868d-e9d6f62d5a0c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:24.403733 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0dd52c-0948-441b-868d-e9d6f62d5a0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:24.422548 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0dd52c-0948-441b-868d-e9d6f62d5a0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:24.425389 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0dd52c-0948-441b-868d-e9d6f62d5a0c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:24.454939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-ce0dd52c-0948-441b-868d-e9d6f62d5a0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:24.454939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:24.454939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:24.454939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:24.454939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:24.454939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:24.454939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:24.454939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:24.454939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:24.454939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:24.454939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:24.454939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:24.454939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:24.454939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:24.454939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:24.454939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:24.456031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:24.456031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:24.456031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:24.456031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:24.456031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:24.456031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:24.456031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:24.456031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:24.456031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:24.456031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:24.456031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:24.456031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:24.456031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:24.456031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:24.456031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:24.456031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:24.456031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:24.457020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:24.457020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:24.457020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:24.457570 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 411/809] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:24.459975 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf0cc9-67d4-4a9d-aeb5-6998d0f85bbf None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:24.460480 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf0cc9-67d4-4a9d-aeb5-6998d0f85bbf None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:24.460932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf0cc9-67d4-4a9d-aeb5-6998d0f85bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:24.468917 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:24.489433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-53cf0cc9-67d4-4a9d-aeb5-6998d0f85bbf None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:24.491598 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-53cf0cc9-67d4-4a9d-aeb5-6998d0f85bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:24.506133 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f68d5f-36ea-48b0-982b-1d23a3eae92e None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:24.548033 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:24.553099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e981eb0-228b-4d65-9ec4-348fcb047912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:24.553660 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e981eb0-228b-4d65-9ec4-348fcb047912 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:24.553929 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e981eb0-228b-4d65-9ec4-348fcb047912 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:24.554193 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e981eb0-228b-4d65-9ec4-348fcb047912 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:24.561816 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e981eb0-228b-4d65-9ec4-348fcb047912 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:24.563285 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e981eb0-228b-4d65-9ec4-348fcb047912 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:24.573163 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 412/810] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:24.574086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf0cc9-67d4-4a9d-aeb5-6998d0f85bbf None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:24.574919 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-6e981eb0-228b-4d65-9ec4-348fcb047912 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:24.574919 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:24.574919 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:24.574919 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:24.574919 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:24.574919 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:24.574919 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:24.574919 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:24.574919 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:24.574919 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:24.574919 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:24.574919 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:24.574919 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:38:24.574919 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:38:24.574919 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:38:24.575799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:38:24.575799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:24.575799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:24.575799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:24.575799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:24.575799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:24.575799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:24.575799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:24.575799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:24.575799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:24.575799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:24.575799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:24.575799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:24.575799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:24.575799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:24.575799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:24.575799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:24.576670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:38:24.576670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:38:24.576670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:38:24.576670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:38:24.576670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:38:24.576670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:38:24.576670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:24.576670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:24.576670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 399/811] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:38:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:24.577154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e981eb0-228b-4d65-9ec4-348fcb047912 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:24.711796 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf0cc9-67d4-4a9d-aeb5-6998d0f85bbf None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:24.719149 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e981eb0-228b-4d65-9ec4-348fcb047912 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:24.757316 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:24.759026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb5e7b-4ed8-4a39-8c33-783e2658adee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:24.759755 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb5e7b-4ed8-4a39-8c33-783e2658adee None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:24.760085 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb5e7b-4ed8-4a39-8c33-783e2658adee None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:24.760456 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb5e7b-4ed8-4a39-8c33-783e2658adee None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:24.761178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb5e7b-4ed8-4a39-8c33-783e2658adee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:24.761521 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb5e7b-4ed8-4a39-8c33-783e2658adee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:24.762957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb5e7b-4ed8-4a39-8c33-783e2658adee None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:24.763507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb5051-4c2a-4dd8-a97c-635c127113ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:24.764146 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb5051-4c2a-4dd8-a97c-635c127113ad None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:24.764421 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb5051-4c2a-4dd8-a97c-635c127113ad None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:24.764808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb5051-4c2a-4dd8-a97c-635c127113ad None admin] PATH_INFO: `/v3/services/network` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:24.765396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb5051-4c2a-4dd8-a97c-635c127113ad None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:24.766762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb5051-4c2a-4dd8-a97c-635c127113ad None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:24.781492 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 413/812] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:38:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:24.781492 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb5e7b-4ed8-4a39-8c33-783e2658adee None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:24.781773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-15fb5051-4c2a-4dd8-a97c-635c127113ad None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 08:38:24.781773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:24.781773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 08:38:24.781773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 08:38:24.781773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 08:38:24.781773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 08:38:24.781773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 08:38:24.781773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:24.781773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 08:38:24.781773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:24.781773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:38:24.781773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:24.781773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:24.781773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:24.781773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:24.781773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:24.781773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:24.783676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:24.783676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:24.783676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:24.783676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:24.783676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:24.783676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:24.783676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 08:38:24.783676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 08:38:24.783676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 08:38:24.783676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 08:38:24.783676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:24.783676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:24.783676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:24.783676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:24.783676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:24.783676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:24.784600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:24.784600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:24.784600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:24.784600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:24.784600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:24.784600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:24.784600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:24.784600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:24.784600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:24.784600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:24.784600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 08:38:24.784600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:24.784600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 08:38:24.784600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:24.784600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 400/813] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:38:24 2022] GET /identity/v3/services/network => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:24.785774 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb5051-4c2a-4dd8-a97c-635c127113ad None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:24.834043 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:24.899805 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb5e7b-4ed8-4a39-8c33-783e2658adee None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:24.900291 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb5051-4c2a-4dd8-a97c-635c127113ad None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:24.937197 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c3f60-70c5-4b95-ba62-d01d50075a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:24.937921 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c3f60-70c5-4b95-ba62-d01d50075a61 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:24.938199 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c3f60-70c5-4b95-ba62-d01d50075a61 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:24.938476 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c3f60-70c5-4b95-ba62-d01d50075a61 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:24.939276 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c3f60-70c5-4b95-ba62-d01d50075a61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:24.939579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c3f60-70c5-4b95-ba62-d01d50075a61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:24.940997 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c3f60-70c5-4b95-ba62-d01d50075a61 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:24.941789 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd7aa9-77c7-4f6c-a7f7-f5fa72ba29d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:24.942566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd7aa9-77c7-4f6c-a7f7-f5fa72ba29d4 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:24.943040 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd7aa9-77c7-4f6c-a7f7-f5fa72ba29d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:24.943389 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd7aa9-77c7-4f6c-a7f7-f5fa72ba29d4 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:24.944343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd7aa9-77c7-4f6c-a7f7-f5fa72ba29d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:24.944772 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd7aa9-77c7-4f6c-a7f7-f5fa72ba29d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:24.946374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd7aa9-77c7-4f6c-a7f7-f5fa72ba29d4 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:24.956676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 414/814] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:24.957681 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c3f60-70c5-4b95-ba62-d01d50075a61 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:24.960439 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 401/815] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:38:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:24.961429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd7aa9-77c7-4f6c-a7f7-f5fa72ba29d4 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:25.021624 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:25.069864 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:25.070128 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd7aa9-77c7-4f6c-a7f7-f5fa72ba29d4 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:25.074024 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c3f60-70c5-4b95-ba62-d01d50075a61 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:25.106702 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104ba00-0b2e-465a-89e5-05cf97dfb194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:25.107301 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104ba00-0b2e-465a-89e5-05cf97dfb194 None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:25.107578 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104ba00-0b2e-465a-89e5-05cf97dfb194 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:25.107844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104ba00-0b2e-465a-89e5-05cf97dfb194 None admin] PATH_INFO: `/v3/projects/543d44293d894a7fbb83401f7a26dd9b/users/df07cffd82c74621855c8b8d7bfa989d/roles/33ac296b51c143e7bb44211175752edb` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:25.110022 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d0938-6d00-4849-84e5-1072ca60bed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:25.110640 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d0938-6d00-4849-84e5-1072ca60bed9 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:25.110940 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d0938-6d00-4849-84e5-1072ca60bed9 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:25.111216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d0938-6d00-4849-84e5-1072ca60bed9 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:25.111852 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d0938-6d00-4849-84e5-1072ca60bed9 None admin] RBAC: Adding query filter params (name=network) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:25.112152 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d0938-6d00-4849-84e5-1072ca60bed9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:25.113525 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d0938-6d00-4849-84e5-1072ca60bed9 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:25.118607 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 415/816] 10.222.0.71 () {56 vars in 1112 bytes} [Fri Aug 26 08:38:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:25.119831 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d0938-6d00-4849-84e5-1072ca60bed9 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:25.127479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1104ba00-0b2e-465a-89e5-05cf97dfb194 None admin] RBAC: Authorizing `identity:create_grant(project_id=543d44293d894a7fbb83401f7a26dd9b, user_id=df07cffd82c74621855c8b8d7bfa989d, role_id=33ac296b51c143e7bb44211175752edb)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:25.128942 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1104ba00-0b2e-465a-89e5-05cf97dfb194 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:25.143911 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 402/817] 10.222.0.71 () {58 vars in 1439 bytes} [Fri Aug 26 08:38:24 2022] PUT /identity/v3/projects/543d44293d894a7fbb83401f7a26dd9b/users/df07cffd82c74621855c8b8d7bfa989d/roles/33ac296b51c143e7bb44211175752edb => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:38:25.144751 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104ba00-0b2e-465a-89e5-05cf97dfb194 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:25.239704 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d0938-6d00-4849-84e5-1072ca60bed9 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:25.267822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104ba00-0b2e-465a-89e5-05cf97dfb194 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:25.277548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:25.280730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5876a7-6d3a-4768-ad45-c62287cf0da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:25.281449 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5876a7-6d3a-4768-ad45-c62287cf0da1 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:25.281769 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5876a7-6d3a-4768-ad45-c62287cf0da1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:25.282038 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5876a7-6d3a-4768-ad45-c62287cf0da1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:25.282666 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5876a7-6d3a-4768-ad45-c62287cf0da1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:25.284202 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5876a7-6d3a-4768-ad45-c62287cf0da1 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:25.296589 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-6f5876a7-6d3a-4768-ad45-c62287cf0da1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 08:38:25.296589 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:25.296589 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 08:38:25.296589 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 08:38:25.296589 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 08:38:25.296589 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 08:38:25.296589 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 08:38:25.296589 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:25.296589 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 08:38:25.296589 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:25.296589 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:38:25.296589 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:25.296589 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:25.296589 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:25.296589 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:25.296589 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:25.296589 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:25.297756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:25.297756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:25.297756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:25.297756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:25.297756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:25.297756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:25.297756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 08:38:25.297756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 08:38:25.297756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 08:38:25.297756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 08:38:25.297756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:25.297756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:25.297756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:25.297756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:25.297756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:25.297756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:25.298699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:25.298699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:25.298699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:25.298699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:25.298699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:25.298699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:25.298699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:25.298699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:25.298699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:25.298699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:25.298699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 08:38:25.298699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:25.298699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 08:38:25.298699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:25.298699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 416/818] 10.222.0.71 () {56 vars in 1129 bytes} [Fri Aug 26 08:38:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:25.299893 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5876a7-6d3a-4768-ad45-c62287cf0da1 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:25.326099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7ae00-233b-4950-bd2f-32ea6f99abbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:25.326773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7ae00-233b-4950-bd2f-32ea6f99abbe None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:25.327233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7ae00-233b-4950-bd2f-32ea6f99abbe None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:25.327408 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7ae00-233b-4950-bd2f-32ea6f99abbe None admin] PATH_INFO: `/v3/users/nova` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:25.336411 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7ae00-233b-4950-bd2f-32ea6f99abbe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:25.337927 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7ae00-233b-4950-bd2f-32ea6f99abbe None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:25.351316 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-f9a7ae00-233b-4950-bd2f-32ea6f99abbe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 08:38:25.351316 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:25.351316 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:25.351316 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:25.351316 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:25.351316 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:25.351316 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:25.351316 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:25.351316 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:25.351316 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:25.351316 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:25.351316 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:25.351316 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:38:25.351316 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:38:25.351316 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:38:25.351316 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:38:25.352652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:25.352652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:25.352652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:38:25.352652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:25.352652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:38:25.352652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:25.352652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:25.352652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:25.352652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:25.352652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:25.352652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:25.352652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:25.352652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:25.352652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:25.352652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:25.352652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:25.352652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:25.352652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:25.353591 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:25.353591 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:25.353591 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:38:25.353591 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:38:25.353591 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:38:25.353591 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:38:25.353591 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:38:25.353591 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:38:25.353591 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 08:38:25.353591 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:25.353591 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 403/819] 10.222.0.71 () {56 vars in 1093 bytes} [Fri Aug 26 08:38:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:25.354196 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7ae00-233b-4950-bd2f-32ea6f99abbe None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:25.455458 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:25.499951 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7ae00-233b-4950-bd2f-32ea6f99abbe None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:25.539560 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca16fb8-59f8-4040-8339-07b49b89ccf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:25.540510 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca16fb8-59f8-4040-8339-07b49b89ccf3 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:25.540784 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca16fb8-59f8-4040-8339-07b49b89ccf3 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:25.541058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca16fb8-59f8-4040-8339-07b49b89ccf3 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:25.541695 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca16fb8-59f8-4040-8339-07b49b89ccf3 None admin] RBAC: Adding query filter params (type=network) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:25.542441 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca16fb8-59f8-4040-8339-07b49b89ccf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:25.543395 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca16fb8-59f8-4040-8339-07b49b89ccf3 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:25.550144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 404/820] 10.222.0.71 () {56 vars in 1112 bytes} [Fri Aug 26 08:38:25 2022] GET /identity/v3/services?type=network => generated 350 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:25.551259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca16fb8-59f8-4040-8339-07b49b89ccf3 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:25.562213 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5876a7-6d3a-4768-ad45-c62287cf0da1 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:25.590189 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:25.606825 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db99498a-8522-49c0-995a-01cf7223df98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:25.608479 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99498a-8522-49c0-995a-01cf7223df98 None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:25.608479 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99498a-8522-49c0-995a-01cf7223df98 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:25.608479 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99498a-8522-49c0-995a-01cf7223df98 None admin] PATH_INFO: `/v3/projects/543d44293d894a7fbb83401f7a26dd9b/users/1723e961375b45b7b3bd99ddeb69683f/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:25.610002 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db99498a-8522-49c0-995a-01cf7223df98 None admin] RBAC: Authorizing `identity:create_grant(project_id=543d44293d894a7fbb83401f7a26dd9b, user_id=1723e961375b45b7b3bd99ddeb69683f, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:25.611574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db99498a-8522-49c0-995a-01cf7223df98 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:25.638394 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 417/821] 10.222.0.71 () {58 vars in 1439 bytes} [Fri Aug 26 08:38:25 2022] PUT /identity/v3/projects/543d44293d894a7fbb83401f7a26dd9b/users/1723e961375b45b7b3bd99ddeb69683f/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:38:25.638394 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db99498a-8522-49c0-995a-01cf7223df98 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:25.690655 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca16fb8-59f8-4040-8339-07b49b89ccf3 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:25.739497 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6b394-46e3-4785-8347-d8ef6fc405b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:25.740421 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6b394-46e3-4785-8347-d8ef6fc405b5 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:25.740701 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6b394-46e3-4785-8347-d8ef6fc405b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:25.740968 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6b394-46e3-4785-8347-d8ef6fc405b5 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:25.741611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6b394-46e3-4785-8347-d8ef6fc405b5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:25.741920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6b394-46e3-4785-8347-d8ef6fc405b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:25.743540 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6b394-46e3-4785-8347-d8ef6fc405b5 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:25.749632 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 405/822] 10.222.0.71 () {56 vars in 1124 bytes} [Fri Aug 26 08:38:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:25.750493 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6b394-46e3-4785-8347-d8ef6fc405b5 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:25.766449 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:25.788751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db99498a-8522-49c0-995a-01cf7223df98 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:25.840258 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82bd381-7f80-41d5-b177-85c287962372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:25.840880 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bd381-7f80-41d5-b177-85c287962372 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:25.841156 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bd381-7f80-41d5-b177-85c287962372 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:25.841427 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bd381-7f80-41d5-b177-85c287962372 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:25.842168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82bd381-7f80-41d5-b177-85c287962372 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:25.842473 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82bd381-7f80-41d5-b177-85c287962372 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:25.846020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82bd381-7f80-41d5-b177-85c287962372 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:25.871777 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 418/823] 10.222.0.71 () {56 vars in 1133 bytes} [Fri Aug 26 08:38:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:25.872981 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82bd381-7f80-41d5-b177-85c287962372 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:25.893779 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:25.902601 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6b394-46e3-4785-8347-d8ef6fc405b5 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:25.955410 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc608635-17bc-44dc-82a0-3167080ef5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:25.956153 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc608635-17bc-44dc-82a0-3167080ef5cc None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:25.956492 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc608635-17bc-44dc-82a0-3167080ef5cc None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:25.956826 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc608635-17bc-44dc-82a0-3167080ef5cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:25.957570 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc608635-17bc-44dc-82a0-3167080ef5cc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:25.959368 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc608635-17bc-44dc-82a0-3167080ef5cc None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:25.992328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 406/824] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:38:25 2022] POST /identity/v3/endpoints => generated 337 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:38:25.993743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc608635-17bc-44dc-82a0-3167080ef5cc None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:26.017101 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82bd381-7f80-41d5-b177-85c287962372 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:26.102300 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455aafb-5e61-410b-bb34-788bd44368f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:26.103365 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455aafb-5e61-410b-bb34-788bd44368f8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:26.103365 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455aafb-5e61-410b-bb34-788bd44368f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:26.103665 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455aafb-5e61-410b-bb34-788bd44368f8 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:26.104265 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3455aafb-5e61-410b-bb34-788bd44368f8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:26.104570 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3455aafb-5e61-410b-bb34-788bd44368f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:26.105968 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3455aafb-5e61-410b-bb34-788bd44368f8 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:26.119342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 419/825] 10.222.0.71 () {56 vars in 1124 bytes} [Fri Aug 26 08:38:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:26.120880 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455aafb-5e61-410b-bb34-788bd44368f8 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:26.130223 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:26.134583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc608635-17bc-44dc-82a0-3167080ef5cc None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:26.178363 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e3dea-4e0f-44ac-b603-5bd148c46576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:26.179764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e3dea-4e0f-44ac-b603-5bd148c46576 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:26.180039 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e3dea-4e0f-44ac-b603-5bd148c46576 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:26.180306 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e3dea-4e0f-44ac-b603-5bd148c46576 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:26.204534 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:26.293702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455aafb-5e61-410b-bb34-788bd44368f8 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:26.314403 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9e3dea-4e0f-44ac-b603-5bd148c46576 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:26.317231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9e3dea-4e0f-44ac-b603-5bd148c46576 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:26.362917 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd8d5f-dd46-4a7c-a086-fef4c193e961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:26.362917 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd8d5f-dd46-4a7c-a086-fef4c193e961 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:26.362917 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd8d5f-dd46-4a7c-a086-fef4c193e961 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:26.362917 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd8d5f-dd46-4a7c-a086-fef4c193e961 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:26.363358 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd8d5f-dd46-4a7c-a086-fef4c193e961 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:26.363358 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd8d5f-dd46-4a7c-a086-fef4c193e961 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:26.409826 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 420/826] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:38:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:38:26.538383 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 407/827] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:38:25 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 08:38:26.544168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd8d5f-dd46-4a7c-a086-fef4c193e961 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:26.632935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:26.686104 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd8d5f-dd46-4a7c-a086-fef4c193e961 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:26.725353 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452b08e-eec0-4353-95d8-c7cff43b4c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:26.726343 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452b08e-eec0-4353-95d8-c7cff43b4c29 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:26.726701 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452b08e-eec0-4353-95d8-c7cff43b4c29 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:26.727142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452b08e-eec0-4353-95d8-c7cff43b4c29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:26.740772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452b08e-eec0-4353-95d8-c7cff43b4c29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:26.742612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452b08e-eec0-4353-95d8-c7cff43b4c29 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:26.759485 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-8452b08e-eec0-4353-95d8-c7cff43b4c29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:26.759485 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:26.759485 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:26.759485 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:26.759485 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:26.759485 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:26.759485 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:26.759485 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:26.759485 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:26.759485 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:26.759485 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:26.759485 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:26.759485 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:26.759485 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:26.759485 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:26.759485 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:26.760529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:26.760529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:26.760529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:26.760529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:26.760529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:26.760529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:26.760529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:26.760529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:26.760529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:26.760529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:26.760529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:26.760529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:26.760529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:26.760529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:26.760529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:26.760529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:26.760529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:26.761477 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:26.761477 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:26.761477 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:26.763871 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 421/828] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:26.772292 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e3dea-4e0f-44ac-b603-5bd148c46576 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:26.830788 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:26.927118 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e3dea-4e0f-44ac-b603-5bd148c46576 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:26.995319 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0a6ab-2d33-4150-b881-0175555c0531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:26.996312 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0a6ab-2d33-4150-b881-0175555c0531 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:26.996785 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0a6ab-2d33-4150-b881-0175555c0531 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:26.997260 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0a6ab-2d33-4150-b881-0175555c0531 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:26.998427 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e0a6ab-2d33-4150-b881-0175555c0531 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:26.998982 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e0a6ab-2d33-4150-b881-0175555c0531 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:27.001506 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e0a6ab-2d33-4150-b881-0175555c0531 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:27.023609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 408/829] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:27.028843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452b08e-eec0-4353-95d8-c7cff43b4c29 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:27.137542 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452b08e-eec0-4353-95d8-c7cff43b4c29 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:27.149962 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:27.191780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6c52a-12a7-405e-bd27-205f32a5c618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:27.192465 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6c52a-12a7-405e-bd27-205f32a5c618 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:27.192739 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6c52a-12a7-405e-bd27-205f32a5c618 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:27.193092 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6c52a-12a7-405e-bd27-205f32a5c618 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:27.200930 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd6c52a-12a7-405e-bd27-205f32a5c618 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:27.202377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd6c52a-12a7-405e-bd27-205f32a5c618 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:27.215807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-2bd6c52a-12a7-405e-bd27-205f32a5c618 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:27.215807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:27.215807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:27.215807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:27.215807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:27.215807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:27.215807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:27.215807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:27.215807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:27.215807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:27.215807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:27.215807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:27.215807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:38:27.215807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:38:27.215807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:38:27.216858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:38:27.216858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:27.216858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:27.216858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:27.216858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:27.216858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:27.216858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:27.216858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:27.216858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:27.216858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:27.216858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:27.216858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:27.216858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:27.216858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:27.216858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:27.216858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:27.216858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:27.217756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:38:27.217756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:38:27.217756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:38:27.217756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:38:27.217756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:38:27.217756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:38:27.217756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:27.217756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:27.217756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 422/830] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:38:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:27.223050 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0a6ab-2d33-4150-b881-0175555c0531 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:27.335125 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0a6ab-2d33-4150-b881-0175555c0531 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:27.349693 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:27.349885 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a99f7-f88a-4014-8e94-64dcbd1ebb9d None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:27.349885 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a99f7-f88a-4014-8e94-64dcbd1ebb9d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:27.349885 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a99f7-f88a-4014-8e94-64dcbd1ebb9d None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:27.350622 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 423/831] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:38:27.356412 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8aafd-ebaf-48f0-bf5d-d7fbe01b14a8 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:27.356988 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8aafd-ebaf-48f0-bf5d-d7fbe01b14a8 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:27.357480 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8aafd-ebaf-48f0-bf5d-d7fbe01b14a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:27.380108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc31138-ee88-4bb0-9f44-fde7b2012e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:27.381112 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc31138-ee88-4bb0-9f44-fde7b2012e58 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:27.381425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-deb8aafd-ebaf-48f0-bf5d-d7fbe01b14a8 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:27.381834 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc31138-ee88-4bb0-9f44-fde7b2012e58 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:27.382314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc31138-ee88-4bb0-9f44-fde7b2012e58 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:27.382664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-deb8aafd-ebaf-48f0-bf5d-d7fbe01b14a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:27.383463 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc31138-ee88-4bb0-9f44-fde7b2012e58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:27.383964 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc31138-ee88-4bb0-9f44-fde7b2012e58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:27.387493 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc31138-ee88-4bb0-9f44-fde7b2012e58 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:27.405122 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 409/832] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:38:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:27.410607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc31138-ee88-4bb0-9f44-fde7b2012e58 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:27.484959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 424/833] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:27.541876 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc31138-ee88-4bb0-9f44-fde7b2012e58 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:27.581226 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37369c01-a15f-4f88-9617-8fe847b10bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:27.581836 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37369c01-a15f-4f88-9617-8fe847b10bfd None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:27.582115 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37369c01-a15f-4f88-9617-8fe847b10bfd None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:27.582389 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37369c01-a15f-4f88-9617-8fe847b10bfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:27.583219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37369c01-a15f-4f88-9617-8fe847b10bfd None admin] RBAC: Adding query filter params (role.id=18c3c8a821b647f7982a955880bd0a9a, scope.project.id=543d44293d894a7fbb83401f7a26dd9b, user.id=f16d1768715a41648af3fe6afadd7477) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:27.583536 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37369c01-a15f-4f88-9617-8fe847b10bfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:27.585153 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37369c01-a15f-4f88-9617-8fe847b10bfd None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:27.592086 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 410/834] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:38:27 2022] GET /identity/v3/role_assignments?user.id=f16d1768715a41648af3fe6afadd7477&scope.project.id=543d44293d894a7fbb83401f7a26dd9b&role.id=18c3c8a821b647f7982a955880bd0a9a => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:27.774972 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8aafd-ebaf-48f0-bf5d-d7fbe01b14a8 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:27.887664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:27.893707 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8aafd-ebaf-48f0-bf5d-d7fbe01b14a8 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:27.939578 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a4bc5-43ad-4fe6-9100-12f932223167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:27.940292 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a4bc5-43ad-4fe6-9100-12f932223167 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:27.940572 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a4bc5-43ad-4fe6-9100-12f932223167 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:27.940845 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a4bc5-43ad-4fe6-9100-12f932223167 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:27.952333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a4bc5-43ad-4fe6-9100-12f932223167 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:38:27.957088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-895a4bc5-43ad-4fe6-9100-12f932223167 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:38:27.957088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:27.957088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:27.957088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:27.957088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:27.957088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:27.957088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:27.957088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:27.957088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:27.957088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:27.957088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:27.957088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:27.957088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:38:27.957088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:38:27.957088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:38:27.957088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:38:27.958093 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:38:27.958093 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:38:27.958093 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:38:27.958093 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:38:27.958093 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:27.958093 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:27.958093 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:27.958093 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:27.958093 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:27.958093 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:27.958093 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:27.958093 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:27.958093 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:27.958093 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:27.958093 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:27.958093 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:27.959305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:27.959305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:27.959305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:27.959305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:27.959305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:38:27.959305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:38:27.959305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:38:27.959305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:38:27.959305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:38:27.959305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:38:27.959305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:38:27.959305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:27.959305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 425/835] 10.222.0.71 () {56 vars in 1102 bytes} [Fri Aug 26 08:38:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:27.965845 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37369c01-a15f-4f88-9617-8fe847b10bfd None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:28.006907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:28.069923 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006349cf-42ce-4f60-9c48-87fe9d84314a None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:28.070551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006349cf-42ce-4f60-9c48-87fe9d84314a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:28.070719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006349cf-42ce-4f60-9c48-87fe9d84314a None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:28.080079 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 426/836] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:38:28.081074 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1986a90-ff5d-4e96-bba2-cfdc0dbf6fe8 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:28.081405 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1986a90-ff5d-4e96-bba2-cfdc0dbf6fe8 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:28.081876 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1986a90-ff5d-4e96-bba2-cfdc0dbf6fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:28.106170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37369c01-a15f-4f88-9617-8fe847b10bfd None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:28.107032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e1986a90-ff5d-4e96-bba2-cfdc0dbf6fe8 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:28.109302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e1986a90-ff5d-4e96-bba2-cfdc0dbf6fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:28.163422 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef10392-6c8b-4057-a7fa-9f0fce933a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:28.163422 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef10392-6c8b-4057-a7fa-9f0fce933a24 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:28.163422 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef10392-6c8b-4057-a7fa-9f0fce933a24 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:28.163422 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef10392-6c8b-4057-a7fa-9f0fce933a24 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:28.163960 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef10392-6c8b-4057-a7fa-9f0fce933a24 None admin] RBAC: Adding query filter params (name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:28.164368 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef10392-6c8b-4057-a7fa-9f0fce933a24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:28.166830 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef10392-6c8b-4057-a7fa-9f0fce933a24 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:28.178173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 411/837] 10.222.0.71 () {56 vars in 1103 bytes} [Fri Aug 26 08:38:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:28.183882 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef10392-6c8b-4057-a7fa-9f0fce933a24 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:28.206934 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 427/838] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:28.296493 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:28.303675 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef10392-6c8b-4057-a7fa-9f0fce933a24 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:28.343261 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deda995-b68f-4efb-9e85-a87e9e621029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:28.343827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deda995-b68f-4efb-9e85-a87e9e621029 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:28.344106 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deda995-b68f-4efb-9e85-a87e9e621029 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:28.344375 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deda995-b68f-4efb-9e85-a87e9e621029 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:28.356756 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deda995-b68f-4efb-9e85-a87e9e621029 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:28.358210 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deda995-b68f-4efb-9e85-a87e9e621029 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:28.374661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-3deda995-b68f-4efb-9e85-a87e9e621029 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:28.374661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:28.374661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:28.374661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:28.374661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:28.374661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:28.374661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:28.374661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:28.374661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:28.374661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:28.374661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:28.374661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:28.374661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:28.374661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:28.374661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:28.374661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:28.375890 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:28.375890 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:28.375890 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:28.375890 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:28.375890 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:28.375890 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:28.375890 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:28.375890 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:28.375890 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:28.375890 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:28.375890 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:28.375890 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:28.375890 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:28.375890 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:28.375890 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:28.375890 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:28.375890 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:28.376765 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:28.376765 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:28.376765 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:28.376765 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 412/839] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:38:28.388043 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1986a90-ff5d-4e96-bba2-cfdc0dbf6fe8 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:28.496112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:28.510403 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1986a90-ff5d-4e96-bba2-cfdc0dbf6fe8 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:28.570665 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb40ba8-f504-467e-888d-390b0071aeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:28.571944 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb40ba8-f504-467e-888d-390b0071aeee None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:28.572528 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb40ba8-f504-467e-888d-390b0071aeee None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:28.573100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb40ba8-f504-467e-888d-390b0071aeee None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:28.574371 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb40ba8-f504-467e-888d-390b0071aeee None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:28.575200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb40ba8-f504-467e-888d-390b0071aeee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:28.577757 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb40ba8-f504-467e-888d-390b0071aeee None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:28.601080 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 428/840] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:28.609108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f31093-0dc1-4119-9c79-ab0ebbd531a9 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:28.609340 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb40ba8-f504-467e-888d-390b0071aeee None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:28.609669 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f31093-0dc1-4119-9c79-ab0ebbd531a9 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:28.610109 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f31093-0dc1-4119-9c79-ab0ebbd531a9 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:28.611844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 413/841] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:28.612958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f31093-0dc1-4119-9c79-ab0ebbd531a9 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:28.769725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb40ba8-f504-467e-888d-390b0071aeee None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:28.770129 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f31093-0dc1-4119-9c79-ab0ebbd531a9 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:28.811443 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e072c5-1e0e-4c72-bcb2-27918b089759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:28.812298 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e072c5-1e0e-4c72-bcb2-27918b089759 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:28.812637 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e072c5-1e0e-4c72-bcb2-27918b089759 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:28.812744 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e072c5-1e0e-4c72-bcb2-27918b089759 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:28.820898 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04ae87-fe80-437c-b0c3-959aead5bab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:28.822388 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04ae87-fe80-437c-b0c3-959aead5bab8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:28.823100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04ae87-fe80-437c-b0c3-959aead5bab8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:28.824111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04ae87-fe80-437c-b0c3-959aead5bab8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:28.834060 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e072c5-1e0e-4c72-bcb2-27918b089759 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:38:28.838436 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04ae87-fe80-437c-b0c3-959aead5bab8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:28.841331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04ae87-fe80-437c-b0c3-959aead5bab8 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:28.842682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-b9e072c5-1e0e-4c72-bcb2-27918b089759 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:38:28.842682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:28.842682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:28.842682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:28.842682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:28.842682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:28.842682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:28.842682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:28.842682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:28.842682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:28.842682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:28.842682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:28.842682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:38:28.842682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:38:28.842682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:38:28.842682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:38:28.844639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:38:28.844639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:38:28.844639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:38:28.844639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:38:28.844639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:28.844639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:28.844639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:28.844639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:28.844639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:28.844639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:28.844639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:28.844639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:28.844639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:28.844639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:28.844639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:28.844639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:28.854907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:28.854907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:28.854907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:28.854907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:28.854907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:38:28.854907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:38:28.854907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:38:28.854907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:38:28.854907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:38:28.854907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:38:28.854907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:38:28.854907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:28.854907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 414/842] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:38:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:28.854907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01458f30-3504-48ae-85cd-eac53227a353 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:28.856425 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01458f30-3504-48ae-85cd-eac53227a353 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:28.856425 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01458f30-3504-48ae-85cd-eac53227a353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:28.861751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-de04ae87-fe80-437c-b0c3-959aead5bab8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 08:38:28.861751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:28.861751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:28.861751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:28.861751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:28.861751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:28.861751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:28.861751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:28.861751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:28.861751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:28.861751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:28.861751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:28.861751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:38:28.861751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:38:28.861751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:38:28.861751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:38:28.864267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:28.864267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:28.864267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:38:28.864267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:28.864267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:38:28.864267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:28.864267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:28.864267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:28.864267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:28.864267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:28.864267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:28.864267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:28.864267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:28.864267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:28.864267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:28.864267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:28.864267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:28.864267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:28.867310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:28.867310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:28.867310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:38:28.867310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:38:28.867310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:38:28.867310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:38:28.867310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:38:28.867310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:38:28.867310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 08:38:28.867310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:28.870945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 429/843] 10.222.0.71 () {56 vars in 1117 bytes} [Fri Aug 26 08:38:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:38:28.871767 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04ae87-fe80-437c-b0c3-959aead5bab8 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:28.877257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-01458f30-3504-48ae-85cd-eac53227a353 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:28.878608 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-01458f30-3504-48ae-85cd-eac53227a353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:28.882339 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:28.885604 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:28.964730 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 415/844] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:28.965975 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01458f30-3504-48ae-85cd-eac53227a353 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:28.993516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04ae87-fe80-437c-b0c3-959aead5bab8 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:29.033973 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee06e8-bbfa-41c0-b61a-eb69206b44aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:29.035770 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee06e8-bbfa-41c0-b61a-eb69206b44aa None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:29.036825 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee06e8-bbfa-41c0-b61a-eb69206b44aa None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:29.037627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee06e8-bbfa-41c0-b61a-eb69206b44aa None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:29.038813 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee06e8-bbfa-41c0-b61a-eb69206b44aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:29.039364 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee06e8-bbfa-41c0-b61a-eb69206b44aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:29.041720 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee06e8-bbfa-41c0-b61a-eb69206b44aa None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:29.058525 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 430/845] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:38:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:29.059809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee06e8-bbfa-41c0-b61a-eb69206b44aa None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:29.100222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01458f30-3504-48ae-85cd-eac53227a353 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:29.139463 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf18c3-e73c-4a71-a186-bfe7fa126b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:29.140261 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf18c3-e73c-4a71-a186-bfe7fa126b7e None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:29.140541 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf18c3-e73c-4a71-a186-bfe7fa126b7e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:29.140951 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf18c3-e73c-4a71-a186-bfe7fa126b7e None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:29.141552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:29.142076 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf18c3-e73c-4a71-a186-bfe7fa126b7e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:29.142944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf18c3-e73c-4a71-a186-bfe7fa126b7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:29.144883 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf18c3-e73c-4a71-a186-bfe7fa126b7e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:29.154814 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:29.172024 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 416/846] 10.222.0.71 () {56 vars in 1149 bytes} [Fri Aug 26 08:38:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:29.173103 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf18c3-e73c-4a71-a186-bfe7fa126b7e None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:29.180081 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee06e8-bbfa-41c0-b61a-eb69206b44aa None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:29.220557 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33225c76-4aea-473f-993d-e90e2374276c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:29.222138 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33225c76-4aea-473f-993d-e90e2374276c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:29.222739 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33225c76-4aea-473f-993d-e90e2374276c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:29.223235 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33225c76-4aea-473f-993d-e90e2374276c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:29.236162 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33225c76-4aea-473f-993d-e90e2374276c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:29.238081 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33225c76-4aea-473f-993d-e90e2374276c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:29.254918 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-33225c76-4aea-473f-993d-e90e2374276c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:29.254918 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:29.254918 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:29.254918 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:29.254918 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:29.254918 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:29.254918 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:29.254918 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:29.254918 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:29.254918 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:29.254918 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:29.254918 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:29.254918 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:29.254918 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:29.254918 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:29.254918 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:29.255969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:29.255969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:29.255969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:29.255969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:29.255969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:29.255969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:29.255969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:29.255969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:29.255969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:29.255969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:29.255969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:29.255969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:29.255969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:29.255969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:29.255969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:29.255969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:29.255969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:29.256865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:29.256865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:29.256865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:29.259038 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 431/847] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:29.260205 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33225c76-4aea-473f-993d-e90e2374276c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:29.294938 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf18c3-e73c-4a71-a186-bfe7fa126b7e None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:29.340530 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87a938-10b3-47c0-9d07-924a10c68404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:29.341838 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87a938-10b3-47c0-9d07-924a10c68404 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:29.342248 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87a938-10b3-47c0-9d07-924a10c68404 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:29.342661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87a938-10b3-47c0-9d07-924a10c68404 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:29.387578 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33225c76-4aea-473f-993d-e90e2374276c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:29.412156 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:29.420336 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:29.429373 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292eb84-0c73-4d2b-b8aa-67c8f6af57b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:29.430933 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292eb84-0c73-4d2b-b8aa-67c8f6af57b6 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:29.431205 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292eb84-0c73-4d2b-b8aa-67c8f6af57b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:29.431435 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f292eb84-0c73-4d2b-b8aa-67c8f6af57b6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:29.432071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292eb84-0c73-4d2b-b8aa-67c8f6af57b6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:29.433531 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f292eb84-0c73-4d2b-b8aa-67c8f6af57b6 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:29.436695 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b87a938-10b3-47c0-9d07-924a10c68404 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:29.438175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b87a938-10b3-47c0-9d07-924a10c68404 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:29.445060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-f292eb84-0c73-4d2b-b8aa-67c8f6af57b6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 08:38:29.445060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:29.445060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 08:38:29.445060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 08:38:29.445060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 08:38:29.445060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 08:38:29.445060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 08:38:29.445060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:29.445060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 08:38:29.445060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:29.445060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:38:29.445060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:29.445060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:29.445060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:29.445060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:29.445060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:29.445060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:29.446308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:29.446308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:29.446308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:29.446308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:29.446308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:29.446308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:29.446308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 08:38:29.446308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 08:38:29.446308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 08:38:29.446308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 08:38:29.446308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:29.446308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:29.446308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:29.446308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:29.446308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:29.446308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:29.447283 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:29.447283 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:29.447283 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:29.447283 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:29.447283 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:29.447283 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:29.447283 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:29.447283 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:29.447283 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:29.447283 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:29.447283 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 08:38:29.447283 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:29.447283 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 08:38:29.447283 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:29.453337 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 432/848] 10.222.0.71 () {56 vars in 1114 bytes} [Fri Aug 26 08:38:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:29.453337 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292eb84-0c73-4d2b-b8aa-67c8f6af57b6 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:29.564401 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 417/849] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:38:29 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 08:38:29.565120 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f292eb84-0c73-4d2b-b8aa-67c8f6af57b6 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:29.565833 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87a938-10b3-47c0-9d07-924a10c68404 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:29.608885 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5daaa17-14f8-4250-a314-5a99f1ce766c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:29.609481 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5daaa17-14f8-4250-a314-5a99f1ce766c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:29.609757 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5daaa17-14f8-4250-a314-5a99f1ce766c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:29.610029 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5daaa17-14f8-4250-a314-5a99f1ce766c None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:29.610692 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5daaa17-14f8-4250-a314-5a99f1ce766c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:29.611026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5daaa17-14f8-4250-a314-5a99f1ce766c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:29.612425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5daaa17-14f8-4250-a314-5a99f1ce766c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:29.625395 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 433/850] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:29.626285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5daaa17-14f8-4250-a314-5a99f1ce766c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:29.683025 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:29.685727 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:29.694468 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87a938-10b3-47c0-9d07-924a10c68404 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:29.746179 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd09fc9-4928-4604-bafc-d099e7a4bf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:29.746809 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd09fc9-4928-4604-bafc-d099e7a4bf06 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:29.747109 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd09fc9-4928-4604-bafc-d099e7a4bf06 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:29.747482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd09fc9-4928-4604-bafc-d099e7a4bf06 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:29.748023 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd09fc9-4928-4604-bafc-d099e7a4bf06 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:29.748321 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd09fc9-4928-4604-bafc-d099e7a4bf06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:29.749766 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd09fc9-4928-4604-bafc-d099e7a4bf06 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:29.751980 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5daaa17-14f8-4250-a314-5a99f1ce766c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:29.755375 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 418/851] 10.222.0.71 () {56 vars in 1114 bytes} [Fri Aug 26 08:38:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:29.756784 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd09fc9-4928-4604-bafc-d099e7a4bf06 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:29.799137 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c36276-7b41-418a-959a-47bbe991752f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:29.799921 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c36276-7b41-418a-959a-47bbe991752f None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:29.800295 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c36276-7b41-418a-959a-47bbe991752f None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:29.800676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c36276-7b41-418a-959a-47bbe991752f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:29.813060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c36276-7b41-418a-959a-47bbe991752f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:29.815279 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c36276-7b41-418a-959a-47bbe991752f None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:29.831966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-b6c36276-7b41-418a-959a-47bbe991752f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:29.831966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:29.831966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:29.831966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:29.831966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:29.831966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:29.831966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:29.831966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:29.831966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:29.831966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:29.831966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:29.831966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:29.831966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:29.831966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:29.831966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:29.831966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:29.833015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:29.833015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:29.833015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:29.833015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:29.833015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:29.833015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:29.833015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:29.833015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:29.833015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:29.833015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:29.833015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:29.833015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:29.833015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:29.833015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:29.833015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:29.833015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:29.833015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:29.833951 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:29.833951 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:29.833951 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:29.833951 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 434/852] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:29.833951 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c36276-7b41-418a-959a-47bbe991752f None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:29.904269 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd09fc9-4928-4604-bafc-d099e7a4bf06 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:29.966685 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e02c20-5a4c-4ca2-bb6b-e018c716a190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:29.967713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e02c20-5a4c-4ca2-bb6b-e018c716a190 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:29.968212 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e02c20-5a4c-4ca2-bb6b-e018c716a190 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:29.968689 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e02c20-5a4c-4ca2-bb6b-e018c716a190 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:29.974463 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:29.978630 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c36276-7b41-418a-959a-47bbe991752f None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:29.979247 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e02c20-5a4c-4ca2-bb6b-e018c716a190 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:29.980798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e02c20-5a4c-4ca2-bb6b-e018c716a190 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:29.996609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-98e02c20-5a4c-4ca2-bb6b-e018c716a190 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 08:38:29.996609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:29.996609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:29.996609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:29.996609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:29.996609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:29.996609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:29.996609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:29.996609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:29.996609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:29.996609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:29.996609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:29.996609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:38:29.996609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:38:29.996609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:38:29.996609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:38:29.999219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:29.999219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:29.999219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:38:29.999219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:29.999219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:38:29.999219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:29.999219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:29.999219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:29.999219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:29.999219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:29.999219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:29.999219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:29.999219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:29.999219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:29.999219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:29.999219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:29.999219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:29.999219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:30.001051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:30.001051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:30.001051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:38:30.001051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:38:30.001051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:38:30.001051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:38:30.001051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:38:30.001051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:38:30.001051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 08:38:30.001051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:30.001051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 419/853] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:38:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:30.001051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e02c20-5a4c-4ca2-bb6b-e018c716a190 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:30.020554 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8969c-0329-4402-a66e-ac864b2cf8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:30.021396 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8969c-0329-4402-a66e-ac864b2cf8fb None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:30.021679 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8969c-0329-4402-a66e-ac864b2cf8fb None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:30.022044 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8969c-0329-4402-a66e-ac864b2cf8fb None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:30.022704 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef8969c-0329-4402-a66e-ac864b2cf8fb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:30.023047 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef8969c-0329-4402-a66e-ac864b2cf8fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:30.024558 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef8969c-0329-4402-a66e-ac864b2cf8fb None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:30.029767 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:30.030020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 435/854] 10.222.0.71 () {56 vars in 1114 bytes} [Fri Aug 26 08:38:29 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:30.031311 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad4e48-3485-42fe-ac41-ec244fe0c157 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:30.031643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad4e48-3485-42fe-ac41-ec244fe0c157 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:30.032001 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad4e48-3485-42fe-ac41-ec244fe0c157 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:30.033634 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 436/855] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:30.034469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad4e48-3485-42fe-ac41-ec244fe0c157 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:30.149602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e02c20-5a4c-4ca2-bb6b-e018c716a190 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:30.164741 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad4e48-3485-42fe-ac41-ec244fe0c157 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:30.192200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bcc0ea-1fc1-4ed3-99b6-51027ccc7032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:30.193427 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcc0ea-1fc1-4ed3-99b6-51027ccc7032 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:30.193708 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcc0ea-1fc1-4ed3-99b6-51027ccc7032 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:30.193980 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcc0ea-1fc1-4ed3-99b6-51027ccc7032 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:30.194954 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bcc0ea-1fc1-4ed3-99b6-51027ccc7032 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:30.195262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bcc0ea-1fc1-4ed3-99b6-51027ccc7032 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:30.196864 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bcc0ea-1fc1-4ed3-99b6-51027ccc7032 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:30.211034 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557e35d-693a-4bca-9e7d-b78f30f74999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:30.211687 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557e35d-693a-4bca-9e7d-b78f30f74999 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:30.211963 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557e35d-693a-4bca-9e7d-b78f30f74999 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:30.212268 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557e35d-693a-4bca-9e7d-b78f30f74999 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:30.213009 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557e35d-693a-4bca-9e7d-b78f30f74999 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:30.213316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557e35d-693a-4bca-9e7d-b78f30f74999 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:30.214678 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 420/856] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:30.215035 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557e35d-693a-4bca-9e7d-b78f30f74999 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:30.216507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4069b1-9066-4e81-a0eb-1af4b123eb2b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:30.216507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4069b1-9066-4e81-a0eb-1af4b123eb2b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:30.216822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4069b1-9066-4e81-a0eb-1af4b123eb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:30.237498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 437/857] 10.222.0.71 () {56 vars in 1135 bytes} [Fri Aug 26 08:38:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:30.238360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557e35d-693a-4bca-9e7d-b78f30f74999 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:30.243761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-cc4069b1-9066-4e81-a0eb-1af4b123eb2b None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:30.245774 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-cc4069b1-9066-4e81-a0eb-1af4b123eb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:30.262979 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:30.336909 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 421/858] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:30.338130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4069b1-9066-4e81-a0eb-1af4b123eb2b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:30.360920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:30.371851 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557e35d-693a-4bca-9e7d-b78f30f74999 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:30.435068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698acf79-ef03-47bb-927f-73ca754328b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:30.436380 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698acf79-ef03-47bb-927f-73ca754328b5 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:30.436866 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698acf79-ef03-47bb-927f-73ca754328b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:30.438001 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698acf79-ef03-47bb-927f-73ca754328b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:30.447937 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698acf79-ef03-47bb-927f-73ca754328b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:30.449808 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698acf79-ef03-47bb-927f-73ca754328b5 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:30.463878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-698acf79-ef03-47bb-927f-73ca754328b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:30.463878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:30.463878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:30.463878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:30.463878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:30.463878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:30.463878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:30.463878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:30.463878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:30.463878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:30.463878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:30.463878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:30.463878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:38:30.463878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:38:30.463878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:38:30.464945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:38:30.464945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:30.464945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:30.464945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:30.464945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:30.464945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:30.464945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:30.464945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:30.464945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:30.464945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:30.464945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:30.464945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:30.464945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:30.464945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:30.464945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:30.464945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:30.464945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:30.466171 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:38:30.466171 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:38:30.466171 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:38:30.466171 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:38:30.466171 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:38:30.466171 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:38:30.466171 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:30.466171 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:30.469124 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 438/859] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:38:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:30.478388 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698acf79-ef03-47bb-927f-73ca754328b5 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:30.478628 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4069b1-9066-4e81-a0eb-1af4b123eb2b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:30.521416 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef34f9c-ae56-4b50-a8f6-6d354846c5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:30.522304 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef34f9c-ae56-4b50-a8f6-6d354846c5a6 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:30.522595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef34f9c-ae56-4b50-a8f6-6d354846c5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:30.522865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef34f9c-ae56-4b50-a8f6-6d354846c5a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:30.570368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:30.592331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698acf79-ef03-47bb-927f-73ca754328b5 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:30.607889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef34f9c-ae56-4b50-a8f6-6d354846c5a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:30.609701 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef34f9c-ae56-4b50-a8f6-6d354846c5a6 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:30.628066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e80550-2acf-4fd3-9ab5-ab74eb44fada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:30.628689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e80550-2acf-4fd3-9ab5-ab74eb44fada None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:30.628970 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e80550-2acf-4fd3-9ab5-ab74eb44fada None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:30.629238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e80550-2acf-4fd3-9ab5-ab74eb44fada None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:30.629939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e80550-2acf-4fd3-9ab5-ab74eb44fada None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:30.630331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e80550-2acf-4fd3-9ab5-ab74eb44fada None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:30.632001 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e80550-2acf-4fd3-9ab5-ab74eb44fada None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:30.632631 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:30.650478 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 439/860] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:38:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:30.651684 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e80550-2acf-4fd3-9ab5-ab74eb44fada None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:30.730823 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 422/861] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:38:30 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 08:38:30.731761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef34f9c-ae56-4b50-a8f6-6d354846c5a6 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:30.763536 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e80550-2acf-4fd3-9ab5-ab74eb44fada None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:30.801371 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940eae75-2afe-4925-a5b4-e7341819b863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:30.802306 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940eae75-2afe-4925-a5b4-e7341819b863 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:30.803010 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940eae75-2afe-4925-a5b4-e7341819b863 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:30.803432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940eae75-2afe-4925-a5b4-e7341819b863 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:30.804480 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940eae75-2afe-4925-a5b4-e7341819b863 None admin] RBAC: Adding query filter params (role.id=33ac296b51c143e7bb44211175752edb, scope.project.id=543d44293d894a7fbb83401f7a26dd9b, user.id=df07cffd82c74621855c8b8d7bfa989d) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:30.804983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940eae75-2afe-4925-a5b4-e7341819b863 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:30.806577 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940eae75-2afe-4925-a5b4-e7341819b863 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:30.813408 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 440/862] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:38:30 2022] GET /identity/v3/role_assignments?user.id=df07cffd82c74621855c8b8d7bfa989d&scope.project.id=543d44293d894a7fbb83401f7a26dd9b&role.id=33ac296b51c143e7bb44211175752edb => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:30.814216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940eae75-2afe-4925-a5b4-e7341819b863 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:30.824925 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:30.861823 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef34f9c-ae56-4b50-a8f6-6d354846c5a6 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:30.879519 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:30.915464 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d4406-5199-4cf1-9566-43399fdbd312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:30.916239 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d4406-5199-4cf1-9566-43399fdbd312 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:30.916607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d4406-5199-4cf1-9566-43399fdbd312 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:30.916944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d4406-5199-4cf1-9566-43399fdbd312 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:30.917649 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d4406-5199-4cf1-9566-43399fdbd312 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:30.919311 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d4406-5199-4cf1-9566-43399fdbd312 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:30.929610 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-ab6d4406-5199-4cf1-9566-43399fdbd312 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 08:38:30.929610 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:30.929610 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 08:38:30.929610 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 08:38:30.929610 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 08:38:30.929610 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 08:38:30.929610 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 08:38:30.929610 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:30.929610 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 08:38:30.929610 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:30.929610 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:38:30.929610 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:30.929610 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:30.929610 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:30.929610 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:30.929610 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:30.930832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:30.930832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:30.930832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:30.930832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:30.930832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:30.930832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:30.930832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:30.930832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 08:38:30.930832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 08:38:30.930832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 08:38:30.930832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 08:38:30.930832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:30.930832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:30.930832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:30.930832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:30.930832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:30.930832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:30.931762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:30.931762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:30.931762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:30.931762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:30.931762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:30.931762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:30.931762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:30.931762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:30.931762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:30.931762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:30.931762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 08:38:30.931762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:30.931762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 08:38:30.931762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:30.936054 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 423/863] 10.222.0.71 () {56 vars in 1132 bytes} [Fri Aug 26 08:38:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:30.937537 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d4406-5199-4cf1-9566-43399fdbd312 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:30.948740 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940eae75-2afe-4925-a5b4-e7341819b863 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:30.992440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45ad36-0e4b-4eca-9f7e-46a245064cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:30.992967 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45ad36-0e4b-4eca-9f7e-46a245064cc8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:30.993277 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45ad36-0e4b-4eca-9f7e-46a245064cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:30.994006 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea45ad36-0e4b-4eca-9f7e-46a245064cc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:31.010206 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45ad36-0e4b-4eca-9f7e-46a245064cc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:31.011914 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea45ad36-0e4b-4eca-9f7e-46a245064cc8 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:31.031563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-ea45ad36-0e4b-4eca-9f7e-46a245064cc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:31.031563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:31.031563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:31.031563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:31.031563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:31.031563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:31.031563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:31.031563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:31.031563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:31.031563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:31.031563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:31.031563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:31.031563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:31.031563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:31.031563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:31.031563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:31.032806 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:31.032806 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:31.032806 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:31.032806 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:31.032806 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:31.032806 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:31.032806 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:31.032806 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:31.032806 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:31.032806 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:31.032806 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:31.032806 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:31.032806 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:31.032806 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:31.032806 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:31.032806 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:31.032806 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:31.033709 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:31.033709 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:31.033709 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:31.033709 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 441/864] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:31.039145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45ad36-0e4b-4eca-9f7e-46a245064cc8 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:31.073034 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d4406-5199-4cf1-9566-43399fdbd312 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:31.133765 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b93489-ff45-4334-b0cc-0139b290102f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:31.134469 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b93489-ff45-4334-b0cc-0139b290102f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:31.134718 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:31.134843 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b93489-ff45-4334-b0cc-0139b290102f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:31.135072 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b93489-ff45-4334-b0cc-0139b290102f None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:31.135952 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b93489-ff45-4334-b0cc-0139b290102f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:31.136185 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b93489-ff45-4334-b0cc-0139b290102f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:31.137725 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b93489-ff45-4334-b0cc-0139b290102f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:31.147025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 424/865] 10.222.0.71 () {56 vars in 1126 bytes} [Fri Aug 26 08:38:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:31.151459 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b93489-ff45-4334-b0cc-0139b290102f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:31.203057 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea45ad36-0e4b-4eca-9f7e-46a245064cc8 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:31.207313 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:31.243975 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b8c97-e414-41d2-b8f0-34581bb86a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:31.244845 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b8c97-e414-41d2-b8f0-34581bb86a9b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:31.245121 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b8c97-e414-41d2-b8f0-34581bb86a9b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:31.245399 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24b8c97-e414-41d2-b8f0-34581bb86a9b None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:31.246039 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24b8c97-e414-41d2-b8f0-34581bb86a9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:31.246669 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24b8c97-e414-41d2-b8f0-34581bb86a9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:31.248581 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24b8c97-e414-41d2-b8f0-34581bb86a9b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:31.266935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 442/866] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:31.271969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b8c97-e414-41d2-b8f0-34581bb86a9b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:31.298802 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b93489-ff45-4334-b0cc-0139b290102f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:31.341907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85db1ca-34b1-4eec-a26f-df81781bf079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:31.342548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85db1ca-34b1-4eec-a26f-df81781bf079 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:31.342817 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85db1ca-34b1-4eec-a26f-df81781bf079 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:31.343193 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85db1ca-34b1-4eec-a26f-df81781bf079 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:31.343870 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85db1ca-34b1-4eec-a26f-df81781bf079 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:31.344168 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85db1ca-34b1-4eec-a26f-df81781bf079 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:31.345852 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85db1ca-34b1-4eec-a26f-df81781bf079 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:31.358953 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 425/867] 10.222.0.71 () {56 vars in 1126 bytes} [Fri Aug 26 08:38:31 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:31.462276 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24b8c97-e414-41d2-b8f0-34581bb86a9b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:31.482655 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:31.504709 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91871b2d-6c11-4631-9cba-acd379d8a439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:31.505280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91871b2d-6c11-4631-9cba-acd379d8a439 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:31.505563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91871b2d-6c11-4631-9cba-acd379d8a439 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:31.505866 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91871b2d-6c11-4631-9cba-acd379d8a439 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:31.514035 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91871b2d-6c11-4631-9cba-acd379d8a439 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:31.515584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91871b2d-6c11-4631-9cba-acd379d8a439 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:31.529091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-91871b2d-6c11-4631-9cba-acd379d8a439 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:31.529091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:31.529091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:31.529091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:31.529091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:31.529091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:31.529091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:31.529091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:31.529091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:31.529091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:31.529091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:31.529091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:31.529091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:38:31.529091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:38:31.529091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:38:31.530080 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:38:31.530080 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:31.530080 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:31.530080 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:31.530080 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:31.530080 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:31.530080 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:31.530080 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:31.530080 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:31.530080 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:31.530080 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:31.530080 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:31.530080 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:31.530080 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:31.530080 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:31.530080 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:31.530080 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:31.530999 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:38:31.530999 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:38:31.530999 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:38:31.530999 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:38:31.530999 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:38:31.530999 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:38:31.530999 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:31.530999 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:31.530999 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 443/868] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:38:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:38:31.536698 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85db1ca-34b1-4eec-a26f-df81781bf079 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:31.660889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85db1ca-34b1-4eec-a26f-df81781bf079 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:31.662609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:31.699832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0af920-733a-480b-ae2e-5c63789f1656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:31.700457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0af920-733a-480b-ae2e-5c63789f1656 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:31.700732 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0af920-733a-480b-ae2e-5c63789f1656 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:31.701002 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0af920-733a-480b-ae2e-5c63789f1656 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:31.701722 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0af920-733a-480b-ae2e-5c63789f1656 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:31.702018 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0af920-733a-480b-ae2e-5c63789f1656 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:31.703515 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0af920-733a-480b-ae2e-5c63789f1656 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:31.723061 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 426/869] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:38:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:31.725875 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91871b2d-6c11-4631-9cba-acd379d8a439 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:31.841540 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91871b2d-6c11-4631-9cba-acd379d8a439 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:31.879698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e39e6fc-ce6f-4072-9752-ae08feaef8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:31.880718 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39e6fc-ce6f-4072-9752-ae08feaef8f6 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:31.881177 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39e6fc-ce6f-4072-9752-ae08feaef8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:31.881620 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39e6fc-ce6f-4072-9752-ae08feaef8f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:31.882535 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e39e6fc-ce6f-4072-9752-ae08feaef8f6 None admin] RBAC: Adding query filter params (role.id=18c3c8a821b647f7982a955880bd0a9a, scope.project.id=543d44293d894a7fbb83401f7a26dd9b, user.id=1723e961375b45b7b3bd99ddeb69683f) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:31.883044 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e39e6fc-ce6f-4072-9752-ae08feaef8f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:31.884705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e39e6fc-ce6f-4072-9752-ae08feaef8f6 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:31.896892 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 444/870] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:38:31 2022] GET /identity/v3/role_assignments?user.id=1723e961375b45b7b3bd99ddeb69683f&scope.project.id=543d44293d894a7fbb83401f7a26dd9b&role.id=18c3c8a821b647f7982a955880bd0a9a => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:32.057336 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a307d-b38c-4167-8b0d-5980d15e7dee None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:32.057724 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a307d-b38c-4167-8b0d-5980d15e7dee None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:32.057924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a307d-b38c-4167-8b0d-5980d15e7dee None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:32.059658 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 427/871] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:32.065503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d3861-7f97-4195-97a0-214955db6893 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:32.066131 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d3861-7f97-4195-97a0-214955db6893 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:32.066406 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d3861-7f97-4195-97a0-214955db6893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:32.089689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e56d3861-7f97-4195-97a0-214955db6893 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:32.090944 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e56d3861-7f97-4195-97a0-214955db6893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:32.136231 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:32.175422 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 445/872] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:32 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:32.447470 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a307d-b38c-4167-8b0d-5980d15e7dee None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:32.560838 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a307d-b38c-4167-8b0d-5980d15e7dee None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:32.599731 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e86d0e5-5a7a-470f-b363-13e9648110b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:32.600345 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86d0e5-5a7a-470f-b363-13e9648110b4 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:32.600625 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86d0e5-5a7a-470f-b363-13e9648110b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:32.600895 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86d0e5-5a7a-470f-b363-13e9648110b4 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:32.601585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e86d0e5-5a7a-470f-b363-13e9648110b4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:32.602997 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e86d0e5-5a7a-470f-b363-13e9648110b4 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:32.620516 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 428/873] 10.222.0.71 () {60 vars in 1141 bytes} [Fri Aug 26 08:38:32 2022] POST /identity/v3/services => generated 251 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:38:33.430671 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58bc41-0d09-4760-9e7e-af61e4e572de None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:33.431426 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58bc41-0d09-4760-9e7e-af61e4e572de None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:33.431426 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58bc41-0d09-4760-9e7e-af61e4e572de None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:33.432866 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 446/874] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:33.438379 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a9317-b8f6-4773-a2be-af099ea16e7b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:33.438670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a9317-b8f6-4773-a2be-af099ea16e7b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:33.438993 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a9317-b8f6-4773-a2be-af099ea16e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:33.460478 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-1a1a9317-b8f6-4773-a2be-af099ea16e7b None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:33.461892 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1a1a9317-b8f6-4773-a2be-af099ea16e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:33.481117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:33.545979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 429/875] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:33 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 08:38:33.610422 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901496c5-8da9-498f-b66b-5262edb61367 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:33.610778 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901496c5-8da9-498f-b66b-5262edb61367 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:33.610983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901496c5-8da9-498f-b66b-5262edb61367 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:33.612595 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 447/876] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:33.624227 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463400c9-6df6-4204-8507-1566b4cb19cb None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:33.624842 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463400c9-6df6-4204-8507-1566b4cb19cb None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:33.625235 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463400c9-6df6-4204-8507-1566b4cb19cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:33.656392 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-463400c9-6df6-4204-8507-1566b4cb19cb None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:33.658338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-463400c9-6df6-4204-8507-1566b4cb19cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:33.766346 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 430/877] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:33 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:33.959421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa568d0-7f56-4ac8-b72e-cc21093c1845 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:33.959421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa568d0-7f56-4ac8-b72e-cc21093c1845 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:33.959421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa568d0-7f56-4ac8-b72e-cc21093c1845 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:33.960147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 448/878] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:33.966258 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04acfdca-0ccb-46c7-8e68-4b292ac808b5 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:33.966670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04acfdca-0ccb-46c7-8e68-4b292ac808b5 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:33.967083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04acfdca-0ccb-46c7-8e68-4b292ac808b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:33.967406 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa568d0-7f56-4ac8-b72e-cc21093c1845 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:33.985884 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-04acfdca-0ccb-46c7-8e68-4b292ac808b5 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:33.987306 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-04acfdca-0ccb-46c7-8e68-4b292ac808b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:34.066913 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 431/879] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:33 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:34.085508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa568d0-7f56-4ac8-b72e-cc21093c1845 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:34.125088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7082a9a-40b5-4f5b-9d95-6062c228df53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:34.125727 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7082a9a-40b5-4f5b-9d95-6062c228df53 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:34.126005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7082a9a-40b5-4f5b-9d95-6062c228df53 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:34.126273 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7082a9a-40b5-4f5b-9d95-6062c228df53 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:34.126966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7082a9a-40b5-4f5b-9d95-6062c228df53 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:34.128383 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7082a9a-40b5-4f5b-9d95-6062c228df53 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:34.146761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 449/880] 10.222.0.71 () {60 vars in 1141 bytes} [Fri Aug 26 08:38:33 2022] POST /identity/v3/services => generated 269 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:38:34.247227 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04acfdca-0ccb-46c7-8e68-4b292ac808b5 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:34.266925 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:34.337375 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7082a9a-40b5-4f5b-9d95-6062c228df53 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:34.409355 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:34.447196 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04acfdca-0ccb-46c7-8e68-4b292ac808b5 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:34.515903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f8c30-88c1-4936-88ad-ddb56af3d588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:34.516658 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f8c30-88c1-4936-88ad-ddb56af3d588 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:34.517132 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f8c30-88c1-4936-88ad-ddb56af3d588 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:34.517599 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f8c30-88c1-4936-88ad-ddb56af3d588 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:34.520442 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7082a9a-40b5-4f5b-9d95-6062c228df53 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:34.536513 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51f8c30-88c1-4936-88ad-ddb56af3d588 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:38:34.545939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-c51f8c30-88c1-4936-88ad-ddb56af3d588 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 08:38:34.545939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:34.545939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:34.545939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:34.545939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:34.545939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:34.545939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:34.545939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:34.545939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:34.545939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:34.545939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:34.545939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:34.545939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:38:34.545939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:38:34.545939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:38:34.545939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:38:34.547256 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:38:34.547256 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:38:34.547256 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:38:34.547256 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:38:34.547256 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:34.547256 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:34.547256 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:34.547256 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:34.547256 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:34.547256 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:34.547256 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:34.547256 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:34.547256 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:34.547256 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:34.547256 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:34.547256 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:34.548684 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:34.548684 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:34.548684 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:34.548684 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:34.548684 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:38:34.548684 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:38:34.548684 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:38:34.548684 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:38:34.548684 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:38:34.548684 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:38:34.548684 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 08:38:34.548684 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:34.551688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 432/881] 10.222.0.71 () {56 vars in 1120 bytes} [Fri Aug 26 08:38:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:34.564150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f8c30-88c1-4936-88ad-ddb56af3d588 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:34.566000 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34190d-a58c-4912-8b4f-61cd19af0a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:34.566656 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34190d-a58c-4912-8b4f-61cd19af0a7b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:34.566999 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34190d-a58c-4912-8b4f-61cd19af0a7b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:34.567269 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34190d-a58c-4912-8b4f-61cd19af0a7b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:34.567864 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34190d-a58c-4912-8b4f-61cd19af0a7b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:34.569750 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34190d-a58c-4912-8b4f-61cd19af0a7b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:34.584439 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-3b34190d-a58c-4912-8b4f-61cd19af0a7b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 08:38:34.584439 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:34.584439 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 08:38:34.584439 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 08:38:34.584439 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 08:38:34.584439 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 08:38:34.584439 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 08:38:34.584439 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:34.584439 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 08:38:34.584439 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:34.584439 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:38:34.584439 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:34.584439 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:34.584439 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:34.584439 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:34.584439 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:34.584439 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:34.585561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:34.585561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:34.585561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:34.585561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:34.585561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:34.585561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:34.585561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 08:38:34.585561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 08:38:34.585561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 08:38:34.585561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 08:38:34.585561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:34.585561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:34.585561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:34.585561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:34.585561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:34.585561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:34.586407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:34.586407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:34.586407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:34.586407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:34.586407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:34.586407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:34.586407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:34.586407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:34.586407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:34.586407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:34.586407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 08:38:34.586407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:34.586407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 08:38:34.586407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:34.589549 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 450/882] 10.222.0.71 () {56 vars in 1126 bytes} [Fri Aug 26 08:38:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:34.596005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34190d-a58c-4912-8b4f-61cd19af0a7b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:34.668572 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:34.737844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f8c30-88c1-4936-88ad-ddb56af3d588 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:34.771013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:34.781651 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebbb77-3847-49fd-9b2d-92b687499a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:34.782281 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebbb77-3847-49fd-9b2d-92b687499a9f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:34.782551 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebbb77-3847-49fd-9b2d-92b687499a9f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:34.782830 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebbb77-3847-49fd-9b2d-92b687499a9f None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:34.783542 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ebbb77-3847-49fd-9b2d-92b687499a9f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:34.783849 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ebbb77-3847-49fd-9b2d-92b687499a9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:34.785233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ebbb77-3847-49fd-9b2d-92b687499a9f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:34.791491 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34190d-a58c-4912-8b4f-61cd19af0a7b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:34.801807 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 433/883] 10.222.0.71 () {56 vars in 1115 bytes} [Fri Aug 26 08:38:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:34.802842 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebbb77-3847-49fd-9b2d-92b687499a9f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:34.831189 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ad42d-3003-4240-aac8-1afbc41fbd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:34.832239 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ad42d-3003-4240-aac8-1afbc41fbd3d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:34.832737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ad42d-3003-4240-aac8-1afbc41fbd3d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:34.833216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ad42d-3003-4240-aac8-1afbc41fbd3d None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:34.834214 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ad42d-3003-4240-aac8-1afbc41fbd3d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:34.834785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ad42d-3003-4240-aac8-1afbc41fbd3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:34.836476 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ad42d-3003-4240-aac8-1afbc41fbd3d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:34.847406 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 451/884] 10.222.0.71 () {56 vars in 1122 bytes} [Fri Aug 26 08:38:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:34.849893 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a6f23-30be-4b1b-9df4-674cf3204734 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:34.850337 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a6f23-30be-4b1b-9df4-674cf3204734 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:34.850680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a6f23-30be-4b1b-9df4-674cf3204734 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:34.852317 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 452/885] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:34.852882 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a6f23-30be-4b1b-9df4-674cf3204734 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:34.931308 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebbb77-3847-49fd-9b2d-92b687499a9f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:34.963885 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:34.974048 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b7285-8190-4191-88d0-bf0b32482693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:34.974864 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b7285-8190-4191-88d0-bf0b32482693 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:34.975161 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b7285-8190-4191-88d0-bf0b32482693 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:34.975430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b7285-8190-4191-88d0-bf0b32482693 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:34.987325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06b7285-8190-4191-88d0-bf0b32482693 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:34.988810 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06b7285-8190-4191-88d0-bf0b32482693 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:35.002237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a6f23-30be-4b1b-9df4-674cf3204734 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:35.004524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-d06b7285-8190-4191-88d0-bf0b32482693 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:35.004524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:35.004524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:35.004524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:35.004524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:35.004524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:35.004524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:35.004524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:35.004524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:35.004524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:35.004524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:35.004524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:35.004524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:35.004524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:35.004524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:35.004524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:35.005464 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:35.005464 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:35.005464 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:35.005464 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:35.005464 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:35.005464 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:35.005464 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:35.005464 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:35.005464 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:35.005464 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:35.005464 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:35.005464 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:35.005464 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:35.005464 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:35.005464 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:35.005464 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:35.005464 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:35.006565 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:35.006565 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:35.006565 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:35.006565 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 434/886] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:35.007321 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3637049-f9dd-401f-8783-5f7478ad9364 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:35.007422 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3637049-f9dd-401f-8783-5f7478ad9364 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:35.007732 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3637049-f9dd-401f-8783-5f7478ad9364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:35.027198 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-f3637049-f9dd-401f-8783-5f7478ad9364 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:35.028307 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f3637049-f9dd-401f-8783-5f7478ad9364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:35.050482 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d29dca-424f-4282-a31e-b7b807a7925b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:35.051195 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d29dca-424f-4282-a31e-b7b807a7925b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:35.051467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d29dca-424f-4282-a31e-b7b807a7925b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:35.051728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d29dca-424f-4282-a31e-b7b807a7925b None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:35.052743 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d29dca-424f-4282-a31e-b7b807a7925b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:35.052915 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d29dca-424f-4282-a31e-b7b807a7925b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:35.054328 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d29dca-424f-4282-a31e-b7b807a7925b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:35.067093 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 453/887] 10.222.0.71 () {56 vars in 1122 bytes} [Fri Aug 26 08:38:34 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:35.068744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d29dca-424f-4282-a31e-b7b807a7925b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:35.098979 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:35.116528 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 435/888] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:35 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:35.209193 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d29dca-424f-4282-a31e-b7b807a7925b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:35.252665 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a058a-8b91-43b0-8634-686767ab7ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:35.253685 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a058a-8b91-43b0-8634-686767ab7ff7 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:35.254181 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a058a-8b91-43b0-8634-686767ab7ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:35.254655 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a058a-8b91-43b0-8634-686767ab7ff7 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:35.255592 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a058a-8b91-43b0-8634-686767ab7ff7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:35.256134 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a058a-8b91-43b0-8634-686767ab7ff7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:35.257853 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a058a-8b91-43b0-8634-686767ab7ff7 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:35.271462 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 454/889] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:35.277633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3637049-f9dd-401f-8783-5f7478ad9364 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:35.398145 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3637049-f9dd-401f-8783-5f7478ad9364 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:35.410707 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:35.424053 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a058a-8b91-43b0-8634-686767ab7ff7 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:35.451658 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b5121d-6bbb-4f33-8e7b-97d2bd4a9fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:35.452303 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5121d-6bbb-4f33-8e7b-97d2bd4a9fe9 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:35.452577 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5121d-6bbb-4f33-8e7b-97d2bd4a9fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:35.452843 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5121d-6bbb-4f33-8e7b-97d2bd4a9fe9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:35.464154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b5121d-6bbb-4f33-8e7b-97d2bd4a9fe9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:35.466296 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b5121d-6bbb-4f33-8e7b-97d2bd4a9fe9 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:35.480614 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-d5b5121d-6bbb-4f33-8e7b-97d2bd4a9fe9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 08:38:35.480614 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:35.480614 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:35.480614 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:35.480614 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:35.480614 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:35.480614 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:35.480614 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:35.480614 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:35.480614 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:35.480614 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:35.480614 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:35.480614 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:38:35.480614 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:38:35.480614 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:38:35.480614 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:38:35.481683 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:35.481683 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:35.481683 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:38:35.481683 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:35.481683 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:38:35.481683 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:35.481683 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:35.481683 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:35.481683 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:35.481683 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:35.481683 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:35.481683 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:35.481683 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:35.481683 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:35.481683 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:35.481683 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:35.481683 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:35.481683 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:35.482661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:35.482661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:35.482661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:38:35.482661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:38:35.482661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:38:35.482661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:38:35.482661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:38:35.482661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:38:35.482661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 08:38:35.482661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:35.485353 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 436/890] 10.222.0.71 () {56 vars in 1117 bytes} [Fri Aug 26 08:38:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:35.491438 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b5121d-6bbb-4f33-8e7b-97d2bd4a9fe9 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:35.531523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:35.567192 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a058a-8b91-43b0-8634-686767ab7ff7 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:35.613939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf7947-d91f-4c16-815f-181a6a2f3b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:35.614751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf7947-d91f-4c16-815f-181a6a2f3b9c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:35.615161 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf7947-d91f-4c16-815f-181a6a2f3b9c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:35.615532 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf7947-d91f-4c16-815f-181a6a2f3b9c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:35.616859 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf7947-d91f-4c16-815f-181a6a2f3b9c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:35.618673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf7947-d91f-4c16-815f-181a6a2f3b9c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:35.632145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-dcbf7947-d91f-4c16-815f-181a6a2f3b9c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 08:38:35.632145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:35.632145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 08:38:35.632145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 08:38:35.632145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 08:38:35.632145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 08:38:35.632145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 08:38:35.632145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:35.632145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 08:38:35.632145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:35.632145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:38:35.632145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:35.632145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:35.632145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:35.632145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:35.632145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:35.632145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:35.633257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:35.633257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:35.633257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:35.633257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:35.633257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:35.633257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:35.633257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 08:38:35.633257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 08:38:35.633257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 08:38:35.633257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 08:38:35.633257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:35.633257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:35.633257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:35.633257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:35.633257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:35.633257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:35.634123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:35.634123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:35.634123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:35.634123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:35.634123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:35.634123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:35.634123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:35.634123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:35.634123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:35.634123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:35.634123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 08:38:35.634123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:35.634123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 08:38:35.634123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:35.635091 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b5121d-6bbb-4f33-8e7b-97d2bd4a9fe9 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:35.635168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 455/891] 10.222.0.71 () {56 vars in 1114 bytes} [Fri Aug 26 08:38:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:38:35.640676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf7947-d91f-4c16-815f-181a6a2f3b9c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:35.682692 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1832fba8-fe45-4b9f-8c2f-52726d285d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:35.683548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832fba8-fe45-4b9f-8c2f-52726d285d90 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:35.683925 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832fba8-fe45-4b9f-8c2f-52726d285d90 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:35.684322 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832fba8-fe45-4b9f-8c2f-52726d285d90 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:35.685185 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1832fba8-fe45-4b9f-8c2f-52726d285d90 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:35.685594 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1832fba8-fe45-4b9f-8c2f-52726d285d90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:35.687171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1832fba8-fe45-4b9f-8c2f-52726d285d90 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:35.708583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 437/892] 10.222.0.71 () {56 vars in 1149 bytes} [Fri Aug 26 08:38:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:35.715345 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1832fba8-fe45-4b9f-8c2f-52726d285d90 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:35.757174 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:35.776660 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf7947-d91f-4c16-815f-181a6a2f3b9c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:35.827574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16213e18-dbec-4ed9-b765-d36fe131974f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:35.828342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16213e18-dbec-4ed9-b765-d36fe131974f None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:35.828659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16213e18-dbec-4ed9-b765-d36fe131974f None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:35.829027 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16213e18-dbec-4ed9-b765-d36fe131974f None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:35.829784 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16213e18-dbec-4ed9-b765-d36fe131974f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:35.830191 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16213e18-dbec-4ed9-b765-d36fe131974f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:35.831714 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16213e18-dbec-4ed9-b765-d36fe131974f None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:35.842406 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 456/893] 10.222.0.71 () {56 vars in 1114 bytes} [Fri Aug 26 08:38:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:35.843464 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16213e18-dbec-4ed9-b765-d36fe131974f None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:35.853832 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:35.862183 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1832fba8-fe45-4b9f-8c2f-52726d285d90 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:35.912493 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6c17f-af82-45e7-9ff7-7e37839ff6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:35.913132 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6c17f-af82-45e7-9ff7-7e37839ff6ff None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:35.913509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6c17f-af82-45e7-9ff7-7e37839ff6ff None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:35.913743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6c17f-af82-45e7-9ff7-7e37839ff6ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:35.963484 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16213e18-dbec-4ed9-b765-d36fe131974f None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:36.001139 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60e0ad-313c-401c-a2fa-0cdcec3f72ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:36.001784 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60e0ad-313c-401c-a2fa-0cdcec3f72ae None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:36.002059 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60e0ad-313c-401c-a2fa-0cdcec3f72ae None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:36.002332 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60e0ad-313c-401c-a2fa-0cdcec3f72ae None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:36.003010 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a60e0ad-313c-401c-a2fa-0cdcec3f72ae None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:36.003317 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a60e0ad-313c-401c-a2fa-0cdcec3f72ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:36.004718 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a60e0ad-313c-401c-a2fa-0cdcec3f72ae None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:36.011502 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6c17f-af82-45e7-9ff7-7e37839ff6ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:36.013310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6c17f-af82-45e7-9ff7-7e37839ff6ff None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:36.015518 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 457/894] 10.222.0.71 () {56 vars in 1114 bytes} [Fri Aug 26 08:38:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:36.016502 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60e0ad-313c-401c-a2fa-0cdcec3f72ae None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:36.049487 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:36.108508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:36.142796 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60e0ad-313c-401c-a2fa-0cdcec3f72ae None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:36.152652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 438/895] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:38:35 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 08:38:36.158045 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6c17f-af82-45e7-9ff7-7e37839ff6ff None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:36.190689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364d493-01e8-492d-aa6c-044ea95f7037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:36.192122 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364d493-01e8-492d-aa6c-044ea95f7037 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:36.192864 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364d493-01e8-492d-aa6c-044ea95f7037 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:36.193465 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364d493-01e8-492d-aa6c-044ea95f7037 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:36.195021 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364d493-01e8-492d-aa6c-044ea95f7037 None admin] RBAC: Adding query filter params (interface=public, service_id=51d6997ebfb94ccd83450359b1b25adc, region_id=RegionOne) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:36.195794 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364d493-01e8-492d-aa6c-044ea95f7037 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:36.198063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364d493-01e8-492d-aa6c-044ea95f7037 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:36.208527 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 458/896] 10.222.0.71 () {56 vars in 1251 bytes} [Fri Aug 26 08:38:36 2022] GET /identity/v3/endpoints?service_id=51d6997ebfb94ccd83450359b1b25adc&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:36.214259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364d493-01e8-492d-aa6c-044ea95f7037 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:36.291016 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6c17f-af82-45e7-9ff7-7e37839ff6ff None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:36.318772 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:36.329995 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20fec1-6a3e-4b45-966c-7e38b3958502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:36.330640 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20fec1-6a3e-4b45-966c-7e38b3958502 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:36.330927 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20fec1-6a3e-4b45-966c-7e38b3958502 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:36.331203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20fec1-6a3e-4b45-966c-7e38b3958502 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:36.340133 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364d493-01e8-492d-aa6c-044ea95f7037 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:36.342119 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20fec1-6a3e-4b45-966c-7e38b3958502 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:36.343615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20fec1-6a3e-4b45-966c-7e38b3958502 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:36.359473 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-5a20fec1-6a3e-4b45-966c-7e38b3958502 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:36.359473 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:36.359473 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:36.359473 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:36.359473 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:36.359473 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:36.359473 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:36.359473 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:36.359473 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:36.359473 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:36.359473 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:36.359473 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:36.359473 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:36.359473 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:36.359473 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:36.359473 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:36.360548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:36.360548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:36.360548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:36.360548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:36.360548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:36.360548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:36.360548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:36.360548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:36.360548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:36.360548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:36.360548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:36.360548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:36.360548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:36.360548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:36.360548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:36.360548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:36.360548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:36.361446 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:36.361446 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:36.361446 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:36.361446 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 439/897] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:36.362307 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca0388-da75-413e-a52a-e216977d7370 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:36.362307 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca0388-da75-413e-a52a-e216977d7370 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:36.362307 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca0388-da75-413e-a52a-e216977d7370 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:36.363745 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 440/898] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:36.366319 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca0388-da75-413e-a52a-e216977d7370 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:36.387300 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5804ce3-57c0-47ed-92be-56b9da829064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:36.387931 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5804ce3-57c0-47ed-92be-56b9da829064 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:36.388212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5804ce3-57c0-47ed-92be-56b9da829064 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:36.388486 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5804ce3-57c0-47ed-92be-56b9da829064 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:36.389128 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5804ce3-57c0-47ed-92be-56b9da829064 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:36.390526 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5804ce3-57c0-47ed-92be-56b9da829064 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:36.391692 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:36.403092 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 459/899] 10.222.0.71 () {56 vars in 1087 bytes} [Fri Aug 26 08:38:36 2022] GET /identity/v3/services => generated 1268 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 08:38:36.404908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8386d69-fe54-41d8-9df9-037e0f882b06 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:36.405191 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8386d69-fe54-41d8-9df9-037e0f882b06 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:36.405801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8386d69-fe54-41d8-9df9-037e0f882b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:36.432081 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-d8386d69-fe54-41d8-9df9-037e0f882b06 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:36.433450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d8386d69-fe54-41d8-9df9-037e0f882b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:36.495701 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca0388-da75-413e-a52a-e216977d7370 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:36.535578 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 460/900] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:36 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:36.548701 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f24424-2277-47e7-ba6c-d90e292c3b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:36.549993 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f24424-2277-47e7-ba6c-d90e292c3b4d None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:36.550527 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f24424-2277-47e7-ba6c-d90e292c3b4d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:36.551119 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f24424-2277-47e7-ba6c-d90e292c3b4d None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:36.552122 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f24424-2277-47e7-ba6c-d90e292c3b4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:36.552695 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f24424-2277-47e7-ba6c-d90e292c3b4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:36.554630 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f24424-2277-47e7-ba6c-d90e292c3b4d None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:36.571205 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 441/901] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:36.577489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8386d69-fe54-41d8-9df9-037e0f882b06 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:36.705386 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8386d69-fe54-41d8-9df9-037e0f882b06 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:36.738221 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:36.752732 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151e805-8b4a-49a0-ac85-634221923e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:36.753449 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151e805-8b4a-49a0-ac85-634221923e02 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:36.753720 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151e805-8b4a-49a0-ac85-634221923e02 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:36.753997 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151e805-8b4a-49a0-ac85-634221923e02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:36.763391 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2151e805-8b4a-49a0-ac85-634221923e02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:36.764931 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2151e805-8b4a-49a0-ac85-634221923e02 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:36.778877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-2151e805-8b4a-49a0-ac85-634221923e02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:36.778877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:36.778877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:36.778877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:36.778877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:36.778877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:36.778877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:36.778877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:36.778877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:36.778877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:36.778877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:36.778877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:36.778877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:38:36.778877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:38:36.778877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:38:36.780099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:38:36.780099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:36.780099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:36.780099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:36.780099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:36.780099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:36.780099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:36.780099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:36.780099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:36.780099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:36.780099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:36.780099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:36.780099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:36.780099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:36.780099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:36.780099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:36.780099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:36.781391 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:38:36.781391 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:38:36.781391 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:38:36.781391 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:38:36.781391 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:38:36.781391 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:38:36.781391 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:36.781391 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:36.781391 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 461/902] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:38:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:36.785968 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f24424-2277-47e7-ba6c-d90e292c3b4d None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:36.837237 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:36.892003 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151e805-8b4a-49a0-ac85-634221923e02 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:36.915074 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f24424-2277-47e7-ba6c-d90e292c3b4d None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:36.962183 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5add2-424c-4141-afd6-8ac65d977d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:36.963947 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5add2-424c-4141-afd6-8ac65d977d99 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:36.964838 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5add2-424c-4141-afd6-8ac65d977d99 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:36.965361 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5add2-424c-4141-afd6-8ac65d977d99 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:36.967115 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5add2-424c-4141-afd6-8ac65d977d99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:36.967517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5add2-424c-4141-afd6-8ac65d977d99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:36.969382 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5add2-424c-4141-afd6-8ac65d977d99 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:36.984087 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 442/903] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:38:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:36.990914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5add2-424c-4141-afd6-8ac65d977d99 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:37.024734 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151e805-8b4a-49a0-ac85-634221923e02 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:37.066333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546f378-ff9b-4071-a838-ddf65e495e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:37.067661 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546f378-ff9b-4071-a838-ddf65e495e2f None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:37.068274 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546f378-ff9b-4071-a838-ddf65e495e2f None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:37.068887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546f378-ff9b-4071-a838-ddf65e495e2f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:37.069798 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8546f378-ff9b-4071-a838-ddf65e495e2f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:37.071224 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8546f378-ff9b-4071-a838-ddf65e495e2f None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:37.080910 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-8546f378-ff9b-4071-a838-ddf65e495e2f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 08:38:37.080910 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:37.080910 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 08:38:37.080910 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 08:38:37.080910 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 08:38:37.080910 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 08:38:37.080910 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 08:38:37.080910 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:37.080910 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 08:38:37.080910 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:37.080910 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:38:37.080910 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:37.080910 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:37.080910 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:37.080910 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:37.080910 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:37.082037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:37.082037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:37.082037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:37.082037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:37.082037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:37.082037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:37.082037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:37.082037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 08:38:37.082037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 08:38:37.082037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 08:38:37.082037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 08:38:37.082037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:37.082037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:37.082037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:37.082037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:37.082037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:37.082037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:37.083782 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:37.083782 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:37.083782 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:37.083782 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:37.083782 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:37.083782 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:37.083782 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:37.083782 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:37.083782 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:37.083782 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:37.083782 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 08:38:37.083782 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:37.083782 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 08:38:37.083782 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:37.083782 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 462/904] 10.222.0.71 () {56 vars in 1132 bytes} [Fri Aug 26 08:38:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:37.084753 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52e9bf-121c-4eab-b98b-cde09ff25739 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:37.084753 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52e9bf-121c-4eab-b98b-cde09ff25739 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:37.084753 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c52e9bf-121c-4eab-b98b-cde09ff25739 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:37.087060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 463/905] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:37.088021 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52e9bf-121c-4eab-b98b-cde09ff25739 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:37.104663 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:37.109386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5add2-424c-4141-afd6-8ac65d977d99 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:37.152501 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa43ec5-c645-47b5-988e-935535bd4a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:37.153115 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa43ec5-c645-47b5-988e-935535bd4a4f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:37.153391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa43ec5-c645-47b5-988e-935535bd4a4f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:37.153661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa43ec5-c645-47b5-988e-935535bd4a4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:37.154398 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa43ec5-c645-47b5-988e-935535bd4a4f None admin] RBAC: Adding query filter params (role.id=f795ff29508a4109be2917083c5ead38, scope.project.id=543d44293d894a7fbb83401f7a26dd9b, user.id=df07cffd82c74621855c8b8d7bfa989d) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:37.154706 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa43ec5-c645-47b5-988e-935535bd4a4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:37.156188 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa43ec5-c645-47b5-988e-935535bd4a4f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:37.166991 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 443/906] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:38:36 2022] GET /identity/v3/role_assignments?user.id=df07cffd82c74621855c8b8d7bfa989d&scope.project.id=543d44293d894a7fbb83401f7a26dd9b&role.id=f795ff29508a4109be2917083c5ead38 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:37.166991 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c721da3-8f30-4a43-b136-5c4db77b650d None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:37.166991 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c721da3-8f30-4a43-b136-5c4db77b650d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:37.166991 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c721da3-8f30-4a43-b136-5c4db77b650d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:37.183474 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:37.197271 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-4c721da3-8f30-4a43-b136-5c4db77b650d None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:37.199522 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4c721da3-8f30-4a43-b136-5c4db77b650d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:37.261852 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c52e9bf-121c-4eab-b98b-cde09ff25739 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:37.315432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 444/907] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:37 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:37.325332 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2e049-526f-421e-bc02-d1fc652f5829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:37.326054 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2e049-526f-421e-bc02-d1fc652f5829 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:37.326359 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2e049-526f-421e-bc02-d1fc652f5829 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:37.326801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2e049-526f-421e-bc02-d1fc652f5829 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:37.327763 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2e049-526f-421e-bc02-d1fc652f5829 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:37.328192 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2e049-526f-421e-bc02-d1fc652f5829 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:37.329787 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f2e049-526f-421e-bc02-d1fc652f5829 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:37.336585 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 464/908] 10.222.0.71 () {56 vars in 1126 bytes} [Fri Aug 26 08:38:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:37.342191 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c721da3-8f30-4a43-b136-5c4db77b650d None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:37.464105 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:37.474376 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c721da3-8f30-4a43-b136-5c4db77b650d None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:37.520162 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874c3b9-bd6a-4412-9874-a196c7c5a2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:37.520784 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874c3b9-bd6a-4412-9874-a196c7c5a2b8 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:37.521191 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874c3b9-bd6a-4412-9874-a196c7c5a2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:37.521679 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8874c3b9-bd6a-4412-9874-a196c7c5a2b8 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:37.522411 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874c3b9-bd6a-4412-9874-a196c7c5a2b8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:37.522718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874c3b9-bd6a-4412-9874-a196c7c5a2b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:37.524536 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8874c3b9-bd6a-4412-9874-a196c7c5a2b8 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:37.532392 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 445/909] 10.222.0.71 () {56 vars in 1126 bytes} [Fri Aug 26 08:38:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:37.538354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2e049-526f-421e-bc02-d1fc652f5829 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:37.621444 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874c3b9-bd6a-4412-9874-a196c7c5a2b8 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:37.672439 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f2e049-526f-421e-bc02-d1fc652f5829 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:37.692409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:37.717691 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c939a-eadc-4842-be45-092e74e86142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:37.718407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c939a-eadc-4842-be45-092e74e86142 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:37.718683 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c939a-eadc-4842-be45-092e74e86142 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:37.718983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c939a-eadc-4842-be45-092e74e86142 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:37.719719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c939a-eadc-4842-be45-092e74e86142 None admin] RBAC: Adding query filter params (interface=public, service_id=b13e184a06614e27aed20144f2f308a9, region_id=RegionOne) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:37.720024 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c939a-eadc-4842-be45-092e74e86142 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:37.721445 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c939a-eadc-4842-be45-092e74e86142 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:37.730171 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 465/910] 10.222.0.71 () {56 vars in 1251 bytes} [Fri Aug 26 08:38:37 2022] GET /identity/v3/endpoints?service_id=b13e184a06614e27aed20144f2f308a9&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:37.736233 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c939a-eadc-4842-be45-092e74e86142 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:37.760326 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8874c3b9-bd6a-4412-9874-a196c7c5a2b8 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:37.811848 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7f341-49ae-40a5-9cb1-b177914790f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:37.812536 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7f341-49ae-40a5-9cb1-b177914790f2 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:37.812715 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7f341-49ae-40a5-9cb1-b177914790f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:37.813127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7f341-49ae-40a5-9cb1-b177914790f2 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:37.813971 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da7f341-49ae-40a5-9cb1-b177914790f2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:37.822711 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da7f341-49ae-40a5-9cb1-b177914790f2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:37.852057 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c939a-eadc-4842-be45-092e74e86142 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:37.859453 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 446/911] 10.222.0.71 () {60 vars in 1141 bytes} [Fri Aug 26 08:38:37 2022] POST /identity/v3/services => generated 241 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:38:37.903574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82133adb-2e18-46e4-a7d4-52b78fe8d208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:37.904208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82133adb-2e18-46e4-a7d4-52b78fe8d208 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:37.904482 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82133adb-2e18-46e4-a7d4-52b78fe8d208 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:37.904749 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82133adb-2e18-46e4-a7d4-52b78fe8d208 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:37.905368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82133adb-2e18-46e4-a7d4-52b78fe8d208 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:37.906794 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82133adb-2e18-46e4-a7d4-52b78fe8d208 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:37.914974 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 466/912] 10.222.0.71 () {56 vars in 1087 bytes} [Fri Aug 26 08:38:37 2022] GET /identity/v3/services => generated 1497 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 08:38:38.504887 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7763d6d-1f61-4363-ae5b-e07240813c9a None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:38.505251 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7763d6d-1f61-4363-ae5b-e07240813c9a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:38.505498 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7763d6d-1f61-4363-ae5b-e07240813c9a None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:38.507306 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 447/913] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:38.513390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2ed89-88a6-43c0-819b-a40c353f6849 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:38.513719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2ed89-88a6-43c0-819b-a40c353f6849 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:38.513946 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2ed89-88a6-43c0-819b-a40c353f6849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:38.533989 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-b5d2ed89-88a6-43c0-819b-a40c353f6849 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:38.535240 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b5d2ed89-88a6-43c0-819b-a40c353f6849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:38.576149 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:38.619060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 467/914] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:38.940655 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7763d6d-1f61-4363-ae5b-e07240813c9a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:38.963181 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:39.088345 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7763d6d-1f61-4363-ae5b-e07240813c9a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:39.127647 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c1201-1689-4ad5-a6e0-718c9c8e0ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:39.128297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c1201-1689-4ad5-a6e0-718c9c8e0ae2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:39.128581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c1201-1689-4ad5-a6e0-718c9c8e0ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:39.128849 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c1201-1689-4ad5-a6e0-718c9c8e0ae2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:39.129450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c1201-1689-4ad5-a6e0-718c9c8e0ae2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:39.131070 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c1201-1689-4ad5-a6e0-718c9c8e0ae2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:39.140754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-992c1201-1689-4ad5-a6e0-718c9c8e0ae2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 08:38:39.140754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:39.140754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 08:38:39.140754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 08:38:39.140754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 08:38:39.140754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 08:38:39.140754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 08:38:39.140754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:39.140754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 08:38:39.140754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:39.140754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:38:39.140754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:39.140754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:39.140754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:39.140754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:39.140754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:39.140754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:39.141827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:39.141827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:39.141827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:39.141827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:39.141827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:39.141827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:39.141827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 08:38:39.141827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 08:38:39.141827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 08:38:39.141827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 08:38:39.141827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:39.141827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:39.141827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:39.141827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:39.141827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:39.141827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:39.142657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:39.142657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:39.142657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:39.142657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:39.142657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:39.142657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:39.142657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:39.142657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:39.142657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:39.142657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:39.142657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 08:38:39.142657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:39.142657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 08:38:39.142657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:39.142657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 448/915] 10.222.0.71 () {56 vars in 1114 bytes} [Fri Aug 26 08:38:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:39.148930 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d2ed89-88a6-43c0-819b-a40c353f6849 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:39.235249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4029b-e150-42d6-b58f-2ff534a9db56 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:39.236029 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4029b-e150-42d6-b58f-2ff534a9db56 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:39.236313 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4029b-e150-42d6-b58f-2ff534a9db56 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:39.237833 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 449/916] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:39.246046 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8343cb6-9bd9-443e-9f3f-cce3b0828f64 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:39.246821 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8343cb6-9bd9-443e-9f3f-cce3b0828f64 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:39.247426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8343cb6-9bd9-443e-9f3f-cce3b0828f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:39.260859 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d2ed89-88a6-43c0-819b-a40c353f6849 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:39.273209 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-f8343cb6-9bd9-443e-9f3f-cce3b0828f64 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:39.275516 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f8343cb6-9bd9-443e-9f3f-cce3b0828f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:39.306605 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9712153-33dc-4037-8757-8ccd2ecaab3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:39.307385 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9712153-33dc-4037-8757-8ccd2ecaab3b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:39.307667 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9712153-33dc-4037-8757-8ccd2ecaab3b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:39.307940 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9712153-33dc-4037-8757-8ccd2ecaab3b None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:39.308582 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9712153-33dc-4037-8757-8ccd2ecaab3b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:39.308886 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9712153-33dc-4037-8757-8ccd2ecaab3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:39.310793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9712153-33dc-4037-8757-8ccd2ecaab3b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:39.318928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 468/917] 10.222.0.71 () {56 vars in 1114 bytes} [Fri Aug 26 08:38:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:39.327504 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9712153-33dc-4037-8757-8ccd2ecaab3b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:39.365677 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 450/918] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:39.368837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:39.443427 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9712153-33dc-4037-8757-8ccd2ecaab3b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:39.480353 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8481e8a-ef72-42e6-a7b4-61154a3f50ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:39.481244 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8481e8a-ef72-42e6-a7b4-61154a3f50ce None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:39.481676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8481e8a-ef72-42e6-a7b4-61154a3f50ce None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:39.482093 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8481e8a-ef72-42e6-a7b4-61154a3f50ce None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:39.482923 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8481e8a-ef72-42e6-a7b4-61154a3f50ce None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:39.483371 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8481e8a-ef72-42e6-a7b4-61154a3f50ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:39.484927 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8481e8a-ef72-42e6-a7b4-61154a3f50ce None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:39.495121 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 469/919] 10.222.0.71 () {56 vars in 1114 bytes} [Fri Aug 26 08:38:39 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:39.497781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8343cb6-9bd9-443e-9f3f-cce3b0828f64 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:39.510796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:39.639419 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8343cb6-9bd9-443e-9f3f-cce3b0828f64 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:39.661286 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8481e8a-ef72-42e6-a7b4-61154a3f50ce None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:39.685080 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3304a24-4e5d-4261-b2dc-3544060d0f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:39.685994 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3304a24-4e5d-4261-b2dc-3544060d0f66 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:39.686437 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3304a24-4e5d-4261-b2dc-3544060d0f66 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:39.687589 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3304a24-4e5d-4261-b2dc-3544060d0f66 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:39.688570 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3304a24-4e5d-4261-b2dc-3544060d0f66 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:39.690487 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3304a24-4e5d-4261-b2dc-3544060d0f66 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:39.727256 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 451/920] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:38:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:38:39.790788 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:39.795816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8481e8a-ef72-42e6-a7b4-61154a3f50ce None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:39.855011 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20f6c1-a3a3-4d92-882b-ce5906b21ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:39.856165 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20f6c1-a3a3-4d92-882b-ce5906b21ca4 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:39.856466 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20f6c1-a3a3-4d92-882b-ce5906b21ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:39.856767 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20f6c1-a3a3-4d92-882b-ce5906b21ca4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:39.877555 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20f6c1-a3a3-4d92-882b-ce5906b21ca4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:38:39.885897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-5c20f6c1-a3a3-4d92-882b-ce5906b21ca4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 08:38:39.885897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:39.885897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:39.885897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:39.885897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:39.885897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:39.885897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:39.885897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:39.885897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:39.885897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:39.885897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:39.885897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:39.885897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:38:39.885897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:38:39.885897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:38:39.885897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:38:39.886959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:38:39.886959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:38:39.886959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:38:39.886959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:38:39.886959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:39.886959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:39.886959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:39.886959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:39.886959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:39.886959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:39.886959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:39.886959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:39.886959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:39.886959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:39.886959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:39.886959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:39.888275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:39.888275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:39.888275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:39.888275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:39.888275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:38:39.888275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:38:39.888275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:38:39.888275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:38:39.888275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:38:39.888275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:38:39.888275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 08:38:39.888275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:39.888275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 470/921] 10.222.0.71 () {56 vars in 1120 bytes} [Fri Aug 26 08:38:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:38:39.895544 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3304a24-4e5d-4261-b2dc-3544060d0f66 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:39.972419 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:40.002479 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1c2d6-d68e-4a48-8bda-a6216cfd39c7 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:40.002789 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1c2d6-d68e-4a48-8bda-a6216cfd39c7 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:40.003072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1c2d6-d68e-4a48-8bda-a6216cfd39c7 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:40.004631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 471/922] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:38:40.010226 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b4bbe-421f-4101-806c-04646756398c None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:40.010513 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b4bbe-421f-4101-806c-04646756398c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:40.010808 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b4bbe-421f-4101-806c-04646756398c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:40.026840 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3304a24-4e5d-4261-b2dc-3544060d0f66 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:40.037810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-177b4bbe-421f-4101-806c-04646756398c None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:40.040069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-177b4bbe-421f-4101-806c-04646756398c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:40.070354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822aef74-2497-4a13-8f2d-7301aea1766a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:40.071005 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822aef74-2497-4a13-8f2d-7301aea1766a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:40.071213 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822aef74-2497-4a13-8f2d-7301aea1766a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:40.071481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822aef74-2497-4a13-8f2d-7301aea1766a None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:40.072163 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822aef74-2497-4a13-8f2d-7301aea1766a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:40.072574 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822aef74-2497-4a13-8f2d-7301aea1766a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:40.073836 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822aef74-2497-4a13-8f2d-7301aea1766a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:40.083937 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 452/923] 10.222.0.71 () {56 vars in 1115 bytes} [Fri Aug 26 08:38:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:40.085552 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82319c-26de-4acd-ac8e-fc2c351f3bfd None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:40.086349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82319c-26de-4acd-ac8e-fc2c351f3bfd None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:40.086630 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82319c-26de-4acd-ac8e-fc2c351f3bfd None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:40.088147 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 453/924] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:40.089807 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82319c-26de-4acd-ac8e-fc2c351f3bfd None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:40.152534 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 472/925] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:40.163219 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab39fce-9e8c-472b-85b5-e12bcdd7ca49 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:40.163219 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab39fce-9e8c-472b-85b5-e12bcdd7ca49 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:40.163219 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab39fce-9e8c-472b-85b5-e12bcdd7ca49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:40.199032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-fab39fce-9e8c-472b-85b5-e12bcdd7ca49 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:40.201223 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-fab39fce-9e8c-472b-85b5-e12bcdd7ca49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:40.264917 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82319c-26de-4acd-ac8e-fc2c351f3bfd None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:40.320110 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:40.331555 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b5f16-7a5b-4f3c-bc67-bb91682d13c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:40.332541 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b5f16-7a5b-4f3c-bc67-bb91682d13c0 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:40.332977 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b5f16-7a5b-4f3c-bc67-bb91682d13c0 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:40.333388 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b5f16-7a5b-4f3c-bc67-bb91682d13c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:40.334971 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 473/926] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:40.346249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b5f16-7a5b-4f3c-bc67-bb91682d13c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:40.348077 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b5f16-7a5b-4f3c-bc67-bb91682d13c0 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:40.368202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-bb4b5f16-7a5b-4f3c-bc67-bb91682d13c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:40.368202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:40.368202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:40.368202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:40.368202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:40.368202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:40.368202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:40.368202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:40.368202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:40.368202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:40.368202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:40.368202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:40.368202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:40.368202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:40.368202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:40.368202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:40.369450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:40.369450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:40.369450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:40.369450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:40.369450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:40.369450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:40.369450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:40.369450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:40.369450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:40.369450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:40.369450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:40.369450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:40.369450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:40.369450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:40.369450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:40.369450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:40.369450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:40.370936 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:40.370936 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:40.370936 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:40.373159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 454/927] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:40.381155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab39fce-9e8c-472b-85b5-e12bcdd7ca49 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:40.395819 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:40.424650 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b5f16-7a5b-4f3c-bc67-bb91682d13c0 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:40.491900 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab39fce-9e8c-472b-85b5-e12bcdd7ca49 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:40.538757 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ce357-67a5-456c-9be8-3eb8c0640bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:40.540053 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ce357-67a5-456c-9be8-3eb8c0640bd8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:40.541041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ce357-67a5-456c-9be8-3eb8c0640bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:40.541961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ce357-67a5-456c-9be8-3eb8c0640bd8 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:40.543001 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ce357-67a5-456c-9be8-3eb8c0640bd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:40.543762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ce357-67a5-456c-9be8-3eb8c0640bd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:40.545926 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ce357-67a5-456c-9be8-3eb8c0640bd8 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:40.564880 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 474/928] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:40.565381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b5f16-7a5b-4f3c-bc67-bb91682d13c0 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:40.572261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ce357-67a5-456c-9be8-3eb8c0640bd8 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:40.614155 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88826b88-59da-4f7f-b3ab-c5c4b90aa476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:40.615560 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88826b88-59da-4f7f-b3ab-c5c4b90aa476 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:40.615843 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88826b88-59da-4f7f-b3ab-c5c4b90aa476 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:40.616155 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88826b88-59da-4f7f-b3ab-c5c4b90aa476 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:40.617313 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88826b88-59da-4f7f-b3ab-c5c4b90aa476 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:40.619102 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88826b88-59da-4f7f-b3ab-c5c4b90aa476 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:40.634602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-88826b88-59da-4f7f-b3ab-c5c4b90aa476 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 08:38:40.634602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:40.634602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 08:38:40.634602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 08:38:40.634602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 08:38:40.634602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 08:38:40.634602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 08:38:40.634602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:40.634602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 08:38:40.634602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:40.634602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:38:40.634602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:40.634602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:40.634602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:40.634602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:40.634602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:40.634602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:40.636600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:40.636600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:40.636600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:40.636600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:40.636600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:40.636600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:40.636600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 08:38:40.636600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 08:38:40.636600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 08:38:40.636600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 08:38:40.636600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:40.636600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:40.636600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:40.636600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:40.636600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:40.636600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:40.639517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:40.639517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:40.639517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:40.639517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:40.639517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:40.639517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:40.639517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:40.639517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:40.639517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:40.639517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:40.639517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 08:38:40.639517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:40.639517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 08:38:40.639517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:40.639517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 455/929] 10.222.0.71 () {56 vars in 1126 bytes} [Fri Aug 26 08:38:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:38:40.641430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88826b88-59da-4f7f-b3ab-c5c4b90aa476 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:40.677899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:40.692100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:40.708446 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ce357-67a5-456c-9be8-3eb8c0640bd8 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:40.745695 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c017eee2-a1ef-4e18-9202-8d2247156295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:40.746520 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017eee2-a1ef-4e18-9202-8d2247156295 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:40.746759 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017eee2-a1ef-4e18-9202-8d2247156295 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:40.747062 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017eee2-a1ef-4e18-9202-8d2247156295 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:40.753323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88826b88-59da-4f7f-b3ab-c5c4b90aa476 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:40.755509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c017eee2-a1ef-4e18-9202-8d2247156295 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:40.756969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c017eee2-a1ef-4e18-9202-8d2247156295 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:40.768450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-c017eee2-a1ef-4e18-9202-8d2247156295 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 08:38:40.768450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:40.768450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:40.768450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:40.768450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:40.768450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:40.768450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:40.768450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:40.768450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:40.768450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:40.768450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:40.768450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:40.768450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:38:40.768450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:38:40.768450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:38:40.768450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:38:40.769772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:40.769772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:40.769772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:38:40.769772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:40.769772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:38:40.769772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:40.769772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:40.769772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:40.769772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:40.769772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:40.769772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:40.769772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:40.769772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:40.769772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:40.769772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:40.769772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:40.769772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:40.769772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:40.770707 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:40.770707 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:40.770707 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:38:40.770707 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:38:40.770707 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:38:40.770707 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:38:40.770707 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:38:40.770707 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:38:40.770707 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 08:38:40.770707 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:40.770707 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 475/930] 10.222.0.71 () {56 vars in 1117 bytes} [Fri Aug 26 08:38:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:40.770707 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c017eee2-a1ef-4e18-9202-8d2247156295 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:40.794429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361955d-0234-4eb0-870d-4b7c70cb9920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:40.795090 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361955d-0234-4eb0-870d-4b7c70cb9920 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:40.795363 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361955d-0234-4eb0-870d-4b7c70cb9920 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:40.795647 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361955d-0234-4eb0-870d-4b7c70cb9920 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:40.796225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361955d-0234-4eb0-870d-4b7c70cb9920 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:40.797594 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361955d-0234-4eb0-870d-4b7c70cb9920 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:40.807323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-a361955d-0234-4eb0-870d-4b7c70cb9920 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 08:38:40.807323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:40.807323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 08:38:40.807323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 08:38:40.807323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 08:38:40.807323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 08:38:40.807323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 08:38:40.807323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:40.807323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 08:38:40.807323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:40.807323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:38:40.807323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:40.807323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:40.807323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:40.807323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:40.807323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:40.808508 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:40.808508 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:40.808508 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:40.808508 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:40.808508 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:40.808508 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:40.808508 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:40.808508 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 08:38:40.808508 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 08:38:40.808508 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 08:38:40.808508 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 08:38:40.808508 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:40.808508 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:40.808508 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:40.808508 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:40.808508 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:40.808508 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:40.809406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:40.809406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:40.809406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:40.809406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:40.809406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:40.809406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:40.809406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:40.809406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:40.809406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:40.809406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:40.809406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 08:38:40.809406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:40.809406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 08:38:40.809406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:40.809406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 456/931] 10.222.0.71 () {56 vars in 1132 bytes} [Fri Aug 26 08:38:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:40.810207 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361955d-0234-4eb0-870d-4b7c70cb9920 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:40.890171 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c017eee2-a1ef-4e18-9202-8d2247156295 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:40.916735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361955d-0234-4eb0-870d-4b7c70cb9920 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:40.926304 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:40.927820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4fee5-bfe0-4e6b-8bdf-3b96f9295638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:40.928430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4fee5-bfe0-4e6b-8bdf-3b96f9295638 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:40.928795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4fee5-bfe0-4e6b-8bdf-3b96f9295638 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:40.929064 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4fee5-bfe0-4e6b-8bdf-3b96f9295638 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:40.929764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4fee5-bfe0-4e6b-8bdf-3b96f9295638 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:40.930073 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4fee5-bfe0-4e6b-8bdf-3b96f9295638 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:40.931515 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4fee5-bfe0-4e6b-8bdf-3b96f9295638 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:40.932822 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:40.938404 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 476/932] 10.222.0.71 () {56 vars in 1122 bytes} [Fri Aug 26 08:38:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:40.939444 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4fee5-bfe0-4e6b-8bdf-3b96f9295638 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:40.961766 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51158d57-bd6b-4351-a28c-f7cb05e1e85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:40.962667 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51158d57-bd6b-4351-a28c-f7cb05e1e85a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:40.962984 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51158d57-bd6b-4351-a28c-f7cb05e1e85a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:40.963266 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51158d57-bd6b-4351-a28c-f7cb05e1e85a None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:40.964203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51158d57-bd6b-4351-a28c-f7cb05e1e85a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:40.964869 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51158d57-bd6b-4351-a28c-f7cb05e1e85a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:40.966326 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51158d57-bd6b-4351-a28c-f7cb05e1e85a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:40.993284 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 457/933] 10.222.0.71 () {56 vars in 1149 bytes} [Fri Aug 26 08:38:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:40.993284 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51158d57-bd6b-4351-a28c-f7cb05e1e85a None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:41.058398 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4fee5-bfe0-4e6b-8bdf-3b96f9295638 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:41.109995 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec3b27-39af-4d89-82a8-8ce510ebdb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:41.110668 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec3b27-39af-4d89-82a8-8ce510ebdb99 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:41.110980 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec3b27-39af-4d89-82a8-8ce510ebdb99 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:41.111357 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec3b27-39af-4d89-82a8-8ce510ebdb99 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:41.111940 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ec3b27-39af-4d89-82a8-8ce510ebdb99 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:41.112245 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ec3b27-39af-4d89-82a8-8ce510ebdb99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:41.113703 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ec3b27-39af-4d89-82a8-8ce510ebdb99 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:41.120109 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 477/934] 10.222.0.71 () {56 vars in 1126 bytes} [Fri Aug 26 08:38:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:41.120795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec3b27-39af-4d89-82a8-8ce510ebdb99 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:41.126377 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51158d57-bd6b-4351-a28c-f7cb05e1e85a None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:41.173731 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3aef3-fd32-4834-9484-8798533fdd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:41.174349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3aef3-fd32-4834-9484-8798533fdd6a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:41.174677 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3aef3-fd32-4834-9484-8798533fdd6a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:41.175029 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3aef3-fd32-4834-9484-8798533fdd6a None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:41.175741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c3aef3-fd32-4834-9484-8798533fdd6a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:41.176053 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c3aef3-fd32-4834-9484-8798533fdd6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:41.177522 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c3aef3-fd32-4834-9484-8798533fdd6a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:41.188634 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 458/935] 10.222.0.71 () {56 vars in 1122 bytes} [Fri Aug 26 08:38:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:41.188634 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3aef3-fd32-4834-9484-8798533fdd6a None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:41.209309 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:41.228501 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:41.259680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec3b27-39af-4d89-82a8-8ce510ebdb99 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:41.300132 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01761d82-c141-43ae-ba49-09a6a180a373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:41.300796 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01761d82-c141-43ae-ba49-09a6a180a373 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:41.301219 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01761d82-c141-43ae-ba49-09a6a180a373 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:41.301325 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01761d82-c141-43ae-ba49-09a6a180a373 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:41.313519 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01761d82-c141-43ae-ba49-09a6a180a373 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:41.315073 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01761d82-c141-43ae-ba49-09a6a180a373 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:41.316919 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3aef3-fd32-4834-9484-8798533fdd6a None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:41.330469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-01761d82-c141-43ae-ba49-09a6a180a373 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:41.330469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:41.330469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:41.330469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:41.330469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:41.330469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:41.330469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:41.330469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:41.330469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:41.330469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:41.330469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:41.330469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:41.330469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:41.330469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:41.330469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:41.330469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:41.331639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:41.331639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:41.331639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:41.331639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:41.331639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:41.331639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:41.331639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:41.331639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:41.331639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:41.331639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:41.331639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:41.331639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:41.331639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:41.331639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:41.331639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:41.331639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:41.331639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:41.332757 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:41.332757 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:41.332757 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:41.336707 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 478/936] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:41.337510 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01761d82-c141-43ae-ba49-09a6a180a373 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:41.361441 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1c7a2-7432-4aeb-8466-5a210f577dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:41.362083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1c7a2-7432-4aeb-8466-5a210f577dbf None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:41.362364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1c7a2-7432-4aeb-8466-5a210f577dbf None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:41.362638 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1c7a2-7432-4aeb-8466-5a210f577dbf None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:41.363359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1c7a2-7432-4aeb-8466-5a210f577dbf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:41.363686 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1c7a2-7432-4aeb-8466-5a210f577dbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:41.365077 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1c7a2-7432-4aeb-8466-5a210f577dbf None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:41.374100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 459/937] 10.222.0.71 () {56 vars in 1126 bytes} [Fri Aug 26 08:38:41 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:41.374874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1c7a2-7432-4aeb-8466-5a210f577dbf None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:41.467321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01761d82-c141-43ae-ba49-09a6a180a373 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:41.484319 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:41.490323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:41.499677 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1c7a2-7432-4aeb-8466-5a210f577dbf None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:41.508630 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5554f-e00d-4741-a01c-76767d3574e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:41.509503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5554f-e00d-4741-a01c-76767d3574e2 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:41.509976 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5554f-e00d-4741-a01c-76767d3574e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:41.510384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5554f-e00d-4741-a01c-76767d3574e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:41.511371 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc5554f-e00d-4741-a01c-76767d3574e2 None admin] RBAC: Adding query filter params (interface=public, service_id=49514efa99d44522a84ed8bc0acb5e74, region_id=RegionOne) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:41.511819 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc5554f-e00d-4741-a01c-76767d3574e2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:41.513372 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc5554f-e00d-4741-a01c-76767d3574e2 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:41.519662 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 479/938] 10.222.0.71 () {56 vars in 1251 bytes} [Fri Aug 26 08:38:41 2022] GET /identity/v3/endpoints?service_id=49514efa99d44522a84ed8bc0acb5e74&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:41.520624 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5554f-e00d-4741-a01c-76767d3574e2 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:41.541477 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b194a1-8dd5-4877-a50c-5e6c55671681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:41.542094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b194a1-8dd5-4877-a50c-5e6c55671681 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:41.542373 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b194a1-8dd5-4877-a50c-5e6c55671681 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:41.542647 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b194a1-8dd5-4877-a50c-5e6c55671681 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:41.543337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b194a1-8dd5-4877-a50c-5e6c55671681 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:41.543636 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b194a1-8dd5-4877-a50c-5e6c55671681 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:41.545021 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b194a1-8dd5-4877-a50c-5e6c55671681 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:41.558932 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 460/939] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:41.559596 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b194a1-8dd5-4877-a50c-5e6c55671681 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:41.652643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5554f-e00d-4741-a01c-76767d3574e2 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:41.674920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b194a1-8dd5-4877-a50c-5e6c55671681 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:41.695795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09a36e-76b7-4a8d-bde6-8c2ffb6763fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:41.697100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09a36e-76b7-4a8d-bde6-8c2ffb6763fd None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:41.697735 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09a36e-76b7-4a8d-bde6-8c2ffb6763fd None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:41.698159 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09a36e-76b7-4a8d-bde6-8c2ffb6763fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:41.699074 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09a36e-76b7-4a8d-bde6-8c2ffb6763fd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:41.700724 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09a36e-76b7-4a8d-bde6-8c2ffb6763fd None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:41.717835 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6188102-28fb-44d9-8091-773776b5ed74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:41.718573 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6188102-28fb-44d9-8091-773776b5ed74 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:41.718951 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6188102-28fb-44d9-8091-773776b5ed74 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:41.719239 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6188102-28fb-44d9-8091-773776b5ed74 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:41.719974 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6188102-28fb-44d9-8091-773776b5ed74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:41.721752 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6188102-28fb-44d9-8091-773776b5ed74 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:41.734959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 461/940] 10.222.0.71 () {56 vars in 1087 bytes} [Fri Aug 26 08:38:41 2022] GET /identity/v3/services => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:38:41.734959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6188102-28fb-44d9-8091-773776b5ed74 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:41.735341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 480/941] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:38:41 2022] POST /identity/v3/endpoints => generated 347 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:38:41.799296 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:41.865226 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6188102-28fb-44d9-8091-773776b5ed74 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:41.921935 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e980d8-3c25-4ce4-9f29-8938d77597e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:41.922620 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e980d8-3c25-4ce4-9f29-8938d77597e2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:41.922956 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e980d8-3c25-4ce4-9f29-8938d77597e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:41.923242 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e980d8-3c25-4ce4-9f29-8938d77597e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:41.931766 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e980d8-3c25-4ce4-9f29-8938d77597e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:41.933195 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e980d8-3c25-4ce4-9f29-8938d77597e2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:41.946280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-56e980d8-3c25-4ce4-9f29-8938d77597e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:41.946280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:41.946280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:41.946280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:41.946280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:41.946280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:41.946280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:41.946280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:41.946280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:41.946280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:41.946280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:41.946280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:41.946280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:38:41.946280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:38:41.946280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:38:41.947343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:38:41.947343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:41.947343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:41.947343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:41.947343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:41.947343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:41.947343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:41.947343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:41.947343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:41.947343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:41.947343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:41.947343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:41.947343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:41.947343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:41.947343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:41.947343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:41.947343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:41.948278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:38:41.948278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:38:41.948278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:38:41.948278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:38:41.948278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:38:41.948278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:38:41.948278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:41.948278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:41.948278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 462/942] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:38:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:41.953833 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09a36e-76b7-4a8d-bde6-8c2ffb6763fd None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:41.991261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:42.054330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6433c6-bf59-4308-b520-835234b96dbc None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:42.054918 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6433c6-bf59-4308-b520-835234b96dbc None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:42.054918 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6433c6-bf59-4308-b520-835234b96dbc None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:42.057769 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 463/943] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:38:42.062260 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a8262a-001d-4110-9e4a-56b12807e101 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:42.062548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a8262a-001d-4110-9e4a-56b12807e101 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:42.062819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a8262a-001d-4110-9e4a-56b12807e101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:42.073803 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09a36e-76b7-4a8d-bde6-8c2ffb6763fd None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:42.083976 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-26a8262a-001d-4110-9e4a-56b12807e101 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:42.085713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-26a8262a-001d-4110-9e4a-56b12807e101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:42.112267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fafcd4-53da-46c1-8283-1374a05c50c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:42.112936 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fafcd4-53da-46c1-8283-1374a05c50c1 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:42.113218 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fafcd4-53da-46c1-8283-1374a05c50c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:42.114117 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fafcd4-53da-46c1-8283-1374a05c50c1 None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:42.114834 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fafcd4-53da-46c1-8283-1374a05c50c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:42.115502 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fafcd4-53da-46c1-8283-1374a05c50c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:42.117309 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fafcd4-53da-46c1-8283-1374a05c50c1 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:42.135165 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 481/944] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:38:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:42.144435 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fafcd4-53da-46c1-8283-1374a05c50c1 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:42.180864 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 464/945] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 08:38:42.270689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:42.280179 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fafcd4-53da-46c1-8283-1374a05c50c1 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:42.321663 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93c0b5-6a01-4f93-98c7-1cd89f9b334a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:42.322366 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93c0b5-6a01-4f93-98c7-1cd89f9b334a None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:42.322653 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93c0b5-6a01-4f93-98c7-1cd89f9b334a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:42.322956 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93c0b5-6a01-4f93-98c7-1cd89f9b334a None admin] PATH_INFO: `/v3/projects/543d44293d894a7fbb83401f7a26dd9b/users/df07cffd82c74621855c8b8d7bfa989d/roles/f795ff29508a4109be2917083c5ead38` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:42.325359 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f93c0b5-6a01-4f93-98c7-1cd89f9b334a None admin] RBAC: Authorizing `identity:create_grant(project_id=543d44293d894a7fbb83401f7a26dd9b, user_id=df07cffd82c74621855c8b8d7bfa989d, role_id=f795ff29508a4109be2917083c5ead38)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:42.326932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f93c0b5-6a01-4f93-98c7-1cd89f9b334a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:42.346361 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 482/946] 10.222.0.71 () {58 vars in 1439 bytes} [Fri Aug 26 08:38:42 2022] PUT /identity/v3/projects/543d44293d894a7fbb83401f7a26dd9b/users/df07cffd82c74621855c8b8d7bfa989d/roles/f795ff29508a4109be2917083c5ead38 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:38:42.460175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a8262a-001d-4110-9e4a-56b12807e101 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:42.464009 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:42.573424 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a8262a-001d-4110-9e4a-56b12807e101 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:42.609595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4225b8-d110-43a9-8031-96adbfa99a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:42.610223 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4225b8-d110-43a9-8031-96adbfa99a77 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:42.610507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4225b8-d110-43a9-8031-96adbfa99a77 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:42.610781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4225b8-d110-43a9-8031-96adbfa99a77 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:42.611518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4225b8-d110-43a9-8031-96adbfa99a77 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:42.613011 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4225b8-d110-43a9-8031-96adbfa99a77 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:42.615760 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: WARNING keystone.server.flask.common [None req-fa4225b8-d110-43a9-8031-96adbfa99a77 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 26 08:38:42.635522 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.sql.core [None req-fa4225b8-d110-43a9-8031-96adbfa99a77 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 26 08:38:42.635522 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 26 08:38:42.635522 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [parameters: {'id': 'ed53634e1e2847dc88849713fefd29c8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 26 08:38:42.635522 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93173) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 26 08:38:42.644425 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-fa4225b8-d110-43a9-8031-96adbfa99a77 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ed53634e1e2847dc88849713fefd29c8.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ed53634e1e2847dc88849713fefd29c8. Aug 26 08:38:42.644425 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:42.644425 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 08:38:42.644425 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 08:38:42.644425 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 08:38:42.644425 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 08:38:42.644425 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 08:38:42.644425 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 08:38:42.644425 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 08:38:42.644425 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application conn.query(q) Aug 26 08:38:42.644425 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 08:38:42.644425 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 08:38:42.644425 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 08:38:42.645234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application result.read() Aug 26 08:38:42.645234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 08:38:42.645234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 08:38:42.645234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 08:38:42.645234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 08:38:42.645234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 08:38:42.645234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 08:38:42.645234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 08:38:42.645234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 08:38:42.645234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 26 08:38:42.645234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:42.645234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 26 08:38:42.645234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:42.645234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:42.645234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 26 08:38:42.645234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 26 08:38:42.645234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 26 08:38:42.645234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 26 08:38:42.646178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 08:38:42.646178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application next(self.gen) Aug 26 08:38:42.646178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 26 08:38:42.646178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application yield resource Aug 26 08:38:42.646178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 08:38:42.646178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application next(self.gen) Aug 26 08:38:42.646178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 26 08:38:42.646178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self.session.rollback() Aug 26 08:38:42.646178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 26 08:38:42.646178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self.force_reraise() Aug 26 08:38:42.646178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 26 08:38:42.646178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise self.value Aug 26 08:38:42.646178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 26 08:38:42.646178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 26 08:38:42.646178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 26 08:38:42.646178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application session.commit() Aug 26 08:38:42.646178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 26 08:38:42.647201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 26 08:38:42.647201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 26 08:38:42.647201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._prepare_impl() Aug 26 08:38:42.647201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 26 08:38:42.647201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self.session.flush() Aug 26 08:38:42.647201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 26 08:38:42.647201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._flush(objects) Aug 26 08:38:42.647201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 26 08:38:42.647201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 26 08:38:42.647201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 26 08:38:42.647201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application compat.raise_( Aug 26 08:38:42.647201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 08:38:42.647201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception Aug 26 08:38:42.647201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 26 08:38:42.647201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application flush_context.execute() Aug 26 08:38:42.647201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 26 08:38:42.647201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rec.execute(self) Aug 26 08:38:42.648135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 26 08:38:42.648135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 26 08:38:42.648135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 26 08:38:42.648135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 26 08:38:42.648135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 26 08:38:42.648135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 26 08:38:42.648135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 26 08:38:42.648135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 26 08:38:42.648135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 26 08:38:42.648135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 26 08:38:42.648135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 26 08:38:42.648135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 26 08:38:42.648135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 26 08:38:42.648135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 26 08:38:42.648135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 26 08:38:42.648961 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 26 08:38:42.648961 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 08:38:42.648961 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception Aug 26 08:38:42.648961 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 08:38:42.648961 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 08:38:42.648961 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 08:38:42.648961 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 08:38:42.648961 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 08:38:42.648961 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 08:38:42.648961 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 08:38:42.648961 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application conn.query(q) Aug 26 08:38:42.648961 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 08:38:42.648961 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 08:38:42.648961 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 08:38:42.648961 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application result.read() Aug 26 08:38:42.648961 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 08:38:42.648961 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 08:38:42.649814 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 08:38:42.649814 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 08:38:42.649814 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 08:38:42.649814 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 08:38:42.649814 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 08:38:42.649814 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 08:38:42.649814 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 26 08:38:42.649814 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 26 08:38:42.649814 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [parameters: {'id': 'ed53634e1e2847dc88849713fefd29c8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 26 08:38:42.649814 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 26 08:38:42.649814 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:42.649814 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:38:42.649814 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:42.649814 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:42.649814 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 26 08:38:42.650688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 26 08:38:42.650688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 26 08:38:42.650688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 26 08:38:42.650688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 26 08:38:42.650688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:42.650688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:38:42.650688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:42.650688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:42.650688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:42.650688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:42.650688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:42.650688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:42.650688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:42.650688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:42.650688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:42.650688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:42.650688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:42.656696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:42.656696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 26 08:38:42.656696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 26 08:38:42.656696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:42.656696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:42.656696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 26 08:38:42.656696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 26 08:38:42.656696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ed53634e1e2847dc88849713fefd29c8. Aug 26 08:38:42.656696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:42.656696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 465/947] 10.222.0.71 () {60 vars in 1140 bytes} [Fri Aug 26 08:38:42 2022] POST /identity/v3/projects => generated 262 bytes in 186 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:38:42.658905 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93c0b5-6a01-4f93-98c7-1cd89f9b334a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:42.773803 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93c0b5-6a01-4f93-98c7-1cd89f9b334a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:42.812314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc6b96-53a0-45b0-91a3-5fa2ae1dd387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:42.813276 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc6b96-53a0-45b0-91a3-5fa2ae1dd387 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:42.813734 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc6b96-53a0-45b0-91a3-5fa2ae1dd387 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:42.814200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc6b96-53a0-45b0-91a3-5fa2ae1dd387 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:42.822305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc6b96-53a0-45b0-91a3-5fa2ae1dd387 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:42.824219 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc6b96-53a0-45b0-91a3-5fa2ae1dd387 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:42.837547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-6bdc6b96-53a0-45b0-91a3-5fa2ae1dd387 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:42.837547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:42.837547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:42.837547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:42.837547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:42.837547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:42.837547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:42.837547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:42.837547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:42.837547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:42.837547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:42.837547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:42.837547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:38:42.837547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:38:42.837547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:38:42.838514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:38:42.838514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:42.838514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:42.838514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:42.838514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:42.838514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:42.838514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:42.838514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:42.838514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:42.838514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:42.838514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:42.838514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:42.838514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:42.838514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:42.838514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:42.838514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:42.838514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:42.839475 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:38:42.839475 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:38:42.839475 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:38:42.839475 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:38:42.839475 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:38:42.839475 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:38:42.839475 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:42.839475 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:42.846075 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 483/948] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:38:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:42.848199 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4225b8-d110-43a9-8031-96adbfa99a77 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:42.918733 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:42.972129 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4225b8-d110-43a9-8031-96adbfa99a77 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:43.013927 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c0ca3-53af-42ec-8736-c6d552c92f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:43.014551 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c0ca3-53af-42ec-8736-c6d552c92f74 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:43.014830 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c0ca3-53af-42ec-8736-c6d552c92f74 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:43.015135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c0ca3-53af-42ec-8736-c6d552c92f74 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:43.015823 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c0ca3-53af-42ec-8736-c6d552c92f74 None admin] RBAC: Adding query filter params (name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:43.016127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c0ca3-53af-42ec-8736-c6d552c92f74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:43.017807 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c0ca3-53af-42ec-8736-c6d552c92f74 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:43.044479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 466/949] 10.222.0.71 () {56 vars in 1112 bytes} [Fri Aug 26 08:38:42 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:43.792808 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190cb93-9dcc-4396-92aa-d8771298384c None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:43.793342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190cb93-9dcc-4396-92aa-d8771298384c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:43.793342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e190cb93-9dcc-4396-92aa-d8771298384c None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:43.794977 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 484/950] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:38:43.800418 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb64b2d-ea1d-437a-8b37-2c1bbab1d461 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:43.800851 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb64b2d-ea1d-437a-8b37-2c1bbab1d461 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:43.801026 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb64b2d-ea1d-437a-8b37-2c1bbab1d461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:43.820991 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-5eb64b2d-ea1d-437a-8b37-2c1bbab1d461 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:43.822535 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5eb64b2d-ea1d-437a-8b37-2c1bbab1d461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:43.913284 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 467/951] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 08:38:44.024156 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e567a-9549-4287-a3b1-a2513f0cd602 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:44.024512 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e567a-9549-4287-a3b1-a2513f0cd602 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:44.024719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e567a-9549-4287-a3b1-a2513f0cd602 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:44.026290 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 485/952] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:44.031760 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730c1ad-9865-4d10-a909-fc8c53a0a9ff None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:44.032284 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730c1ad-9865-4d10-a909-fc8c53a0a9ff None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:44.032772 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a730c1ad-9865-4d10-a909-fc8c53a0a9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:44.053292 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-a730c1ad-9865-4d10-a909-fc8c53a0a9ff None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:44.054819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a730c1ad-9865-4d10-a909-fc8c53a0a9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:44.108028 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:44.139016 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 468/953] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:44.184544 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e567a-9549-4287-a3b1-a2513f0cd602 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:44.204393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:44.320823 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e567a-9549-4287-a3b1-a2513f0cd602 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:44.353744 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb88bf5-b88f-41a6-8ea2-4a253d47faa5 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:44.354139 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb88bf5-b88f-41a6-8ea2-4a253d47faa5 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:44.354420 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb88bf5-b88f-41a6-8ea2-4a253d47faa5 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:44.356731 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 469/954] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:44.361379 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c540b-6aee-4f19-80b4-8de159e682b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:44.362029 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c540b-6aee-4f19-80b4-8de159e682b0 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:44.362305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c540b-6aee-4f19-80b4-8de159e682b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:44.362575 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c540b-6aee-4f19-80b4-8de159e682b0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:44.363204 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c540b-6aee-4f19-80b4-8de159e682b0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:44.364059 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5d851-a1ec-4af7-a60b-dc9a718cb644 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:44.364348 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5d851-a1ec-4af7-a60b-dc9a718cb644 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:44.364581 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c540b-6aee-4f19-80b4-8de159e682b0 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:44.364683 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5d851-a1ec-4af7-a60b-dc9a718cb644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:44.374625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-641c540b-6aee-4f19-80b4-8de159e682b0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 08:38:44.374625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:44.374625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 08:38:44.374625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 08:38:44.374625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 08:38:44.374625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 08:38:44.374625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 08:38:44.374625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:44.374625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 08:38:44.374625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:44.374625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:38:44.374625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:44.374625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:44.374625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:44.374625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:44.374625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:44.374625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:44.375912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:44.375912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:44.375912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:44.375912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:44.375912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:44.375912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:44.375912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 08:38:44.375912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 08:38:44.375912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 08:38:44.375912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 08:38:44.375912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:44.375912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:44.375912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:44.375912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:44.375912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:44.375912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:44.376839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:44.376839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:44.376839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:44.376839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:44.376839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:44.376839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:44.376839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:44.376839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:44.376839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:44.376839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:44.376839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 08:38:44.376839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:44.376839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 08:38:44.376839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:44.376839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 486/955] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:38:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:44.381657 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c540b-6aee-4f19-80b4-8de159e682b0 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:44.396225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-29f5d851-a1ec-4af7-a60b-dc9a718cb644 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:44.397997 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-29f5d851-a1ec-4af7-a60b-dc9a718cb644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:44.466655 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:44.479223 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 470/956] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:44.480186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5d851-a1ec-4af7-a60b-dc9a718cb644 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:44.501302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c540b-6aee-4f19-80b4-8de159e682b0 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:44.552307 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7fc584-baf8-4601-8ebe-4a9c49861150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:44.553466 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fc584-baf8-4601-8ebe-4a9c49861150 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:44.554778 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fc584-baf8-4601-8ebe-4a9c49861150 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:44.555151 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fc584-baf8-4601-8ebe-4a9c49861150 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:44.555935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7fc584-baf8-4601-8ebe-4a9c49861150 None admin] RBAC: Adding query filter params (name=compute) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:44.556244 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7fc584-baf8-4601-8ebe-4a9c49861150 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:44.558511 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7fc584-baf8-4601-8ebe-4a9c49861150 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:44.565507 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 487/957] 10.222.0.71 () {56 vars in 1112 bytes} [Fri Aug 26 08:38:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:44.571234 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7fc584-baf8-4601-8ebe-4a9c49861150 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:44.618278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5d851-a1ec-4af7-a60b-dc9a718cb644 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:44.631837 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:44.661201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3ea47-6b3f-419c-9d39-3880944f0612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:44.661871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3ea47-6b3f-419c-9d39-3880944f0612 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:44.662152 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3ea47-6b3f-419c-9d39-3880944f0612 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:44.662511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3ea47-6b3f-419c-9d39-3880944f0612 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:44.663261 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3ea47-6b3f-419c-9d39-3880944f0612 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:44.664765 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3ea47-6b3f-419c-9d39-3880944f0612 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:44.674950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-01c3ea47-6b3f-419c-9d39-3880944f0612 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 08:38:44.674950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:44.674950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 08:38:44.674950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 08:38:44.674950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 08:38:44.674950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 08:38:44.674950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 08:38:44.674950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:44.674950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 08:38:44.674950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:44.674950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:38:44.674950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:44.674950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:44.674950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:44.674950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:44.674950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:44.674950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:44.676036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:44.676036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:44.676036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:44.676036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:44.676036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:44.676036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:44.676036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 08:38:44.676036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 08:38:44.676036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 08:38:44.676036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 08:38:44.676036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:44.676036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:44.676036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:44.676036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:44.676036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:44.676036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:44.676898 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:44.676898 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:44.676898 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:44.676898 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:44.676898 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:44.676898 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:44.676898 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:44.676898 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:44.676898 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:44.676898 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:44.676898 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 08:38:44.676898 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:44.676898 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 08:38:44.676898 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:44.676898 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 471/958] 10.222.0.71 () {56 vars in 1126 bytes} [Fri Aug 26 08:38:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:44.683938 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3ea47-6b3f-419c-9d39-3880944f0612 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:44.712596 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7fc584-baf8-4601-8ebe-4a9c49861150 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:44.761715 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277aa8a-0861-47d6-a43e-8b33d4b23a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:44.762345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277aa8a-0861-47d6-a43e-8b33d4b23a70 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:44.762616 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277aa8a-0861-47d6-a43e-8b33d4b23a70 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:44.762908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277aa8a-0861-47d6-a43e-8b33d4b23a70 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:44.763571 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277aa8a-0861-47d6-a43e-8b33d4b23a70 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:44.763877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277aa8a-0861-47d6-a43e-8b33d4b23a70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:44.765260 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277aa8a-0861-47d6-a43e-8b33d4b23a70 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:44.770270 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 488/959] 10.222.0.71 () {56 vars in 1112 bytes} [Fri Aug 26 08:38:44 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:44.771533 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277aa8a-0861-47d6-a43e-8b33d4b23a70 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:44.787786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:44.814466 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3ea47-6b3f-419c-9d39-3880944f0612 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:44.874650 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f845461-8347-44de-8c19-4e38e4349959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:44.875663 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f845461-8347-44de-8c19-4e38e4349959 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:44.876140 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f845461-8347-44de-8c19-4e38e4349959 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:44.876668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f845461-8347-44de-8c19-4e38e4349959 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:44.877531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f845461-8347-44de-8c19-4e38e4349959 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:44.878067 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f845461-8347-44de-8c19-4e38e4349959 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:44.879815 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f845461-8347-44de-8c19-4e38e4349959 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:44.887315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 472/960] 10.222.0.71 () {56 vars in 1122 bytes} [Fri Aug 26 08:38:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:44.893209 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f845461-8347-44de-8c19-4e38e4349959 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:44.940214 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277aa8a-0861-47d6-a43e-8b33d4b23a70 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:44.946239 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:44.997027 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e9aab-230f-41ae-9434-a2f2f95a60f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:44.997729 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e9aab-230f-41ae-9434-a2f2f95a60f4 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:44.998061 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e9aab-230f-41ae-9434-a2f2f95a60f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:44.998387 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e9aab-230f-41ae-9434-a2f2f95a60f4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:45.011820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8e9aab-230f-41ae-9434-a2f2f95a60f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:38:45.017509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-2b8e9aab-230f-41ae-9434-a2f2f95a60f4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 08:38:45.017509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:45.017509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:45.017509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:45.017509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:45.017509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:45.017509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:45.017509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:45.017509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:45.017509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:45.017509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:45.017509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:45.017509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:38:45.017509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:38:45.017509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:38:45.017509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:38:45.018564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:38:45.018564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:38:45.018564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:38:45.018564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:38:45.018564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:45.018564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:45.018564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:45.018564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:45.018564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:45.018564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:45.018564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:45.018564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:45.018564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:45.018564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:45.018564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:45.018564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:45.019469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:45.019469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:45.019469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:45.019469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:45.019469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:38:45.019469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:38:45.019469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:38:45.019469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:38:45.019469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:38:45.019469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:38:45.019469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 08:38:45.019469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:45.019469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 489/961] 10.222.0.71 () {56 vars in 1120 bytes} [Fri Aug 26 08:38:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:45.025243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e9aab-230f-41ae-9434-a2f2f95a60f4 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:45.040260 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f845461-8347-44de-8c19-4e38e4349959 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:45.081496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebc76d-43e2-4ff4-9b7b-0d919962e3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:45.082560 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebc76d-43e2-4ff4-9b7b-0d919962e3d3 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:45.083072 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebc76d-43e2-4ff4-9b7b-0d919962e3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:45.083545 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebc76d-43e2-4ff4-9b7b-0d919962e3d3 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:45.084550 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebc76d-43e2-4ff4-9b7b-0d919962e3d3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:45.085157 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebc76d-43e2-4ff4-9b7b-0d919962e3d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:45.086771 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebc76d-43e2-4ff4-9b7b-0d919962e3d3 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:45.093350 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 473/962] 10.222.0.71 () {56 vars in 1122 bytes} [Fri Aug 26 08:38:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:45.099284 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebc76d-43e2-4ff4-9b7b-0d919962e3d3 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:45.106466 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:45.158251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e9aab-230f-41ae-9434-a2f2f95a60f4 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:45.209320 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7004e0-e92b-4c87-b9b1-7e6d35449f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:45.209953 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7004e0-e92b-4c87-b9b1-7e6d35449f3a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:45.210236 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7004e0-e92b-4c87-b9b1-7e6d35449f3a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:45.210511 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7004e0-e92b-4c87-b9b1-7e6d35449f3a None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:45.211252 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7004e0-e92b-4c87-b9b1-7e6d35449f3a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:45.211558 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7004e0-e92b-4c87-b9b1-7e6d35449f3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:45.212955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7004e0-e92b-4c87-b9b1-7e6d35449f3a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:45.226635 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 490/963] 10.222.0.71 () {56 vars in 1115 bytes} [Fri Aug 26 08:38:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:45.229455 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7004e0-e92b-4c87-b9b1-7e6d35449f3a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:45.254616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebc76d-43e2-4ff4-9b7b-0d919962e3d3 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:45.256929 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:45.322186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553f093-28d2-4d08-933e-7ff2accc4ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:45.323377 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553f093-28d2-4d08-933e-7ff2accc4ca1 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:45.323753 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553f093-28d2-4d08-933e-7ff2accc4ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:45.324030 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553f093-28d2-4d08-933e-7ff2accc4ca1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:45.324808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9553f093-28d2-4d08-933e-7ff2accc4ca1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:45.326745 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9553f093-28d2-4d08-933e-7ff2accc4ca1 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:45.356870 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 474/964] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:38:45 2022] POST /identity/v3/endpoints => generated 349 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:38:45.357204 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8b6a7-4f4a-4845-b04b-38f7767f05e9 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:45.357369 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8b6a7-4f4a-4845-b04b-38f7767f05e9 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:45.357649 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8b6a7-4f4a-4845-b04b-38f7767f05e9 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:45.359202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 475/965] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:45.370861 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126a0cc-2555-4119-8e54-3b9aa4fb306e None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:45.370861 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126a0cc-2555-4119-8e54-3b9aa4fb306e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:45.370861 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126a0cc-2555-4119-8e54-3b9aa4fb306e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:45.387360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7004e0-e92b-4c87-b9b1-7e6d35449f3a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:45.392128 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-5126a0cc-2555-4119-8e54-3b9aa4fb306e None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:45.393414 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5126a0cc-2555-4119-8e54-3b9aa4fb306e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:45.420343 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:45.439261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a4d9d0-b3cd-4db8-ac89-55711d11ea82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:45.440532 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4d9d0-b3cd-4db8-ac89-55711d11ea82 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:45.441034 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4d9d0-b3cd-4db8-ac89-55711d11ea82 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:45.441512 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4d9d0-b3cd-4db8-ac89-55711d11ea82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:45.457186 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a4d9d0-b3cd-4db8-ac89-55711d11ea82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:45.459569 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a4d9d0-b3cd-4db8-ac89-55711d11ea82 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:45.480849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-b2a4d9d0-b3cd-4db8-ac89-55711d11ea82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:45.480849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:45.480849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:45.480849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:45.480849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:45.480849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:45.480849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:45.480849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:45.480849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:45.480849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:45.480849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:45.480849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:45.480849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:45.480849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:45.480849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:45.480849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:45.481869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:45.481869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:45.481869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:45.481869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:45.481869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:45.481869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:45.481869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:45.481869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:45.481869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:45.481869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:45.481869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:45.481869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:45.481869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:45.481869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:45.481869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:45.481869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:45.481869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:45.482798 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:45.482798 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:45.482798 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:45.489582 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 491/966] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:45.490434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a4d9d0-b3cd-4db8-ac89-55711d11ea82 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:45.496673 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 476/967] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 08:38:45.603445 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a4d9d0-b3cd-4db8-ac89-55711d11ea82 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:45.639672 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23628589-59f0-45b7-a256-9426d75f99df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:45.640644 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23628589-59f0-45b7-a256-9426d75f99df None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:45.641180 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23628589-59f0-45b7-a256-9426d75f99df None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:45.641650 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23628589-59f0-45b7-a256-9426d75f99df None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:45.642555 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23628589-59f0-45b7-a256-9426d75f99df None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:45.643089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23628589-59f0-45b7-a256-9426d75f99df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:45.644664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23628589-59f0-45b7-a256-9426d75f99df None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:45.657999 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 492/968] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:45.664189 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126a0cc-2555-4119-8e54-3b9aa4fb306e None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:45.764973 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:45.769560 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23628589-59f0-45b7-a256-9426d75f99df None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:45.781927 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126a0cc-2555-4119-8e54-3b9aa4fb306e None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:45.821331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79679f40-25fd-4101-9d4e-bdd799b2dcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:45.822048 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79679f40-25fd-4101-9d4e-bdd799b2dcba None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:45.822132 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79679f40-25fd-4101-9d4e-bdd799b2dcba None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:45.822367 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79679f40-25fd-4101-9d4e-bdd799b2dcba None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:45.830598 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79679f40-25fd-4101-9d4e-bdd799b2dcba None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:45.832362 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79679f40-25fd-4101-9d4e-bdd799b2dcba None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:45.845808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-79679f40-25fd-4101-9d4e-bdd799b2dcba None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 08:38:45.845808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:45.845808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:45.845808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:45.845808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:45.845808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:45.845808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:45.845808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:45.845808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:45.845808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:45.845808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:45.845808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:45.845808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:38:45.845808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:38:45.845808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:38:45.845808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:38:45.846986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:45.846986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:45.846986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:38:45.846986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:45.846986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:38:45.846986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:45.846986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:45.846986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:45.846986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:45.846986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:45.846986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:45.846986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:45.846986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:45.846986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:45.846986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:45.846986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:45.846986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:45.846986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:45.848044 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:45.848044 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:45.848044 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:38:45.848044 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:38:45.848044 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:38:45.848044 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:38:45.848044 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:38:45.848044 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:38:45.848044 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 08:38:45.848044 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:45.848614 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:45.850688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 477/969] 10.222.0.71 () {56 vars in 1117 bytes} [Fri Aug 26 08:38:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:45.856792 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79679f40-25fd-4101-9d4e-bdd799b2dcba None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:45.895971 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23628589-59f0-45b7-a256-9426d75f99df None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:45.937364 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe2b5b8-6596-42ac-a7a8-83ac0af896f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:45.938339 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2b5b8-6596-42ac-a7a8-83ac0af896f8 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:45.938792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2b5b8-6596-42ac-a7a8-83ac0af896f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:45.939279 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2b5b8-6596-42ac-a7a8-83ac0af896f8 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:45.940129 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe2b5b8-6596-42ac-a7a8-83ac0af896f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:45.941882 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe2b5b8-6596-42ac-a7a8-83ac0af896f8 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:45.946358 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: WARNING keystone.server.flask.common [None req-3fe2b5b8-6596-42ac-a7a8-83ac0af896f8 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 26 08:38:45.960160 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.sql.core [None req-3fe2b5b8-6596-42ac-a7a8-83ac0af896f8 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 26 08:38:45.960160 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 26 08:38:45.960160 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [parameters: {'user_id': '92aa6ca31089431f9e02a7e74e3b32da', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 26 08:38:45.960160 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93172) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 26 08:38:45.981088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-3fe2b5b8-6596-42ac-a7a8-83ac0af896f8 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 26 08:38:45.981088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:45.981088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 08:38:45.981088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 08:38:45.981088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 08:38:45.981088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 08:38:45.981088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 08:38:45.981088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 08:38:45.981088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 08:38:45.981088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application conn.query(q) Aug 26 08:38:45.981088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 08:38:45.981088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 08:38:45.981088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 08:38:45.981088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application result.read() Aug 26 08:38:45.981088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 08:38:45.982063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 08:38:45.982063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 08:38:45.982063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 08:38:45.982063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 08:38:45.982063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 08:38:45.982063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 08:38:45.982063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 08:38:45.982063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 26 08:38:45.982063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:45.982063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 26 08:38:45.982063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:45.982063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:45.982063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 26 08:38:45.982063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 26 08:38:45.982063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 26 08:38:45.982063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 26 08:38:45.982063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 08:38:45.982063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application next(self.gen) Aug 26 08:38:45.983498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 26 08:38:45.983498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application yield resource Aug 26 08:38:45.983498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 08:38:45.983498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application next(self.gen) Aug 26 08:38:45.983498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 26 08:38:45.983498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self.session.rollback() Aug 26 08:38:45.983498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 26 08:38:45.983498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self.force_reraise() Aug 26 08:38:45.983498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 26 08:38:45.983498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise self.value Aug 26 08:38:45.983498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 26 08:38:45.983498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 26 08:38:45.983498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 26 08:38:45.983498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application session.commit() Aug 26 08:38:45.983498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 26 08:38:45.983498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 26 08:38:45.983498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 26 08:38:45.985060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._prepare_impl() Aug 26 08:38:45.985060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 26 08:38:45.985060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self.session.flush() Aug 26 08:38:45.985060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 26 08:38:45.985060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._flush(objects) Aug 26 08:38:45.985060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 26 08:38:45.985060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 26 08:38:45.985060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 26 08:38:45.985060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application compat.raise_( Aug 26 08:38:45.985060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 08:38:45.985060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception Aug 26 08:38:45.985060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 26 08:38:45.985060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application flush_context.execute() Aug 26 08:38:45.985060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 26 08:38:45.985060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rec.execute(self) Aug 26 08:38:45.985060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 26 08:38:45.985060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 26 08:38:45.986844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 26 08:38:45.986844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 26 08:38:45.986844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 26 08:38:45.986844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 26 08:38:45.986844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 26 08:38:45.986844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 26 08:38:45.986844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 26 08:38:45.986844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 26 08:38:45.986844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 26 08:38:45.986844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 26 08:38:45.986844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 26 08:38:45.986844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 26 08:38:45.986844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 26 08:38:45.986844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 26 08:38:45.986844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 08:38:45.986844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception Aug 26 08:38:45.988955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 08:38:45.988955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 08:38:45.988955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 08:38:45.988955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 08:38:45.988955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 08:38:45.988955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 08:38:45.988955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 08:38:45.988955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application conn.query(q) Aug 26 08:38:45.988955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 08:38:45.988955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 08:38:45.988955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 08:38:45.988955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application result.read() Aug 26 08:38:45.988955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 08:38:45.988955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 08:38:45.988955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 08:38:45.988955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 08:38:45.991100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 08:38:45.991100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 08:38:45.991100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 08:38:45.991100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 08:38:45.991100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 26 08:38:45.991100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 26 08:38:45.991100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [parameters: {'user_id': '92aa6ca31089431f9e02a7e74e3b32da', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 26 08:38:45.991100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 26 08:38:45.991100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:45.991100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:38:45.991100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:45.991100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:45.991100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:45.991100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:45.991100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:45.993013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:45.993013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:45.993013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:45.993013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:45.993013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:45.993013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:45.993013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:45.993013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 26 08:38:45.993013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 26 08:38:45.993013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:45.993013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:45.993013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:38:45.993013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:45.993013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:38:45.993013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:45.993013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 26 08:38:45.993013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 26 08:38:45.997263 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 26 08:38:45.997263 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 26 08:38:45.997263 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 26 08:38:45.997263 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 26 08:38:45.997263 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 26 08:38:45.997263 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:45.999337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79679f40-25fd-4101-9d4e-bdd799b2dcba None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:46.002677 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 493/970] 10.222.0.71 () {60 vars in 1131 bytes} [Fri Aug 26 08:38:45 2022] POST /identity/v3/users => generated 160 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:46.004170 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe2b5b8-6596-42ac-a7a8-83ac0af896f8 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:46.022754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9dfac-3aa9-4887-a79f-198993a219cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:46.023323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9dfac-3aa9-4887-a79f-198993a219cd None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:46.023603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9dfac-3aa9-4887-a79f-198993a219cd None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:46.023874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9dfac-3aa9-4887-a79f-198993a219cd None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:46.024698 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef9dfac-3aa9-4887-a79f-198993a219cd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:46.025005 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef9dfac-3aa9-4887-a79f-198993a219cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:46.026485 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef9dfac-3aa9-4887-a79f-198993a219cd None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:46.047429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 478/971] 10.222.0.71 () {56 vars in 1149 bytes} [Fri Aug 26 08:38:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:46.054822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9dfac-3aa9-4887-a79f-198993a219cd None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:46.067038 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:46.137819 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:46.141816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe2b5b8-6596-42ac-a7a8-83ac0af896f8 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:46.189020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4581c76-d040-4c54-8cb4-6830a6c03555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:46.190067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4581c76-d040-4c54-8cb4-6830a6c03555 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:46.190554 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4581c76-d040-4c54-8cb4-6830a6c03555 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:46.191055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4581c76-d040-4c54-8cb4-6830a6c03555 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:46.195932 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9dfac-3aa9-4887-a79f-198993a219cd None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:46.200426 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4581c76-d040-4c54-8cb4-6830a6c03555 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:46.202264 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4581c76-d040-4c54-8cb4-6830a6c03555 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:46.218178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-e4581c76-d040-4c54-8cb4-6830a6c03555 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 08:38:46.218178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:46.218178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:46.218178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:46.218178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:46.218178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:46.218178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:46.218178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:46.218178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:46.218178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:46.218178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:46.218178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:46.218178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:38:46.218178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:38:46.218178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:38:46.218178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:38:46.220115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:46.220115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:46.220115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:38:46.220115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:46.220115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:38:46.220115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:46.220115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:46.220115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:46.220115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:46.220115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:46.220115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:46.220115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:46.220115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:46.220115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:46.220115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:46.220115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:46.220115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:46.220115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:46.221857 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:46.221857 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:46.221857 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:38:46.221857 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:38:46.221857 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:38:46.221857 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:38:46.221857 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:38:46.221857 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:38:46.221857 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 08:38:46.221857 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:46.224542 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 494/972] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:38:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:46.225954 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4581c76-d040-4c54-8cb4-6830a6c03555 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:46.251965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903ce47-5a89-4890-941a-f3664cda6299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:46.253457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903ce47-5a89-4890-941a-f3664cda6299 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:46.254296 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903ce47-5a89-4890-941a-f3664cda6299 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:46.255098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903ce47-5a89-4890-941a-f3664cda6299 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:46.351052 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903ce47-5a89-4890-941a-f3664cda6299 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:46.352740 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903ce47-5a89-4890-941a-f3664cda6299 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:46.363855 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4581c76-d040-4c54-8cb4-6830a6c03555 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:46.367273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:46.412856 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd62c9ce-f232-4d7b-8d25-82ed6be99e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:46.413503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62c9ce-f232-4d7b-8d25-82ed6be99e75 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:46.413783 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62c9ce-f232-4d7b-8d25-82ed6be99e75 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:46.414082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62c9ce-f232-4d7b-8d25-82ed6be99e75 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:46.415101 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd62c9ce-f232-4d7b-8d25-82ed6be99e75 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:46.415408 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd62c9ce-f232-4d7b-8d25-82ed6be99e75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:46.417599 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd62c9ce-f232-4d7b-8d25-82ed6be99e75 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:46.475429 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 495/973] 10.222.0.71 () {56 vars in 1101 bytes} [Fri Aug 26 08:38:46 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:46.483067 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 479/974] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:38:46 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 08:38:46.484775 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd62c9ce-f232-4d7b-8d25-82ed6be99e75 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:46.540138 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:46.609593 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd62c9ce-f232-4d7b-8d25-82ed6be99e75 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:46.648104 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc6372-bd0b-4574-896b-76cf47c3a255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:46.648758 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc6372-bd0b-4574-896b-76cf47c3a255 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:46.649036 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc6372-bd0b-4574-896b-76cf47c3a255 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:46.649308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc6372-bd0b-4574-896b-76cf47c3a255 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:46.660652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafc6372-bd0b-4574-896b-76cf47c3a255 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:46.662575 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafc6372-bd0b-4574-896b-76cf47c3a255 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:46.680883 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-aafc6372-bd0b-4574-896b-76cf47c3a255 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:46.680883 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:46.680883 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:46.680883 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:46.680883 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:46.680883 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:46.680883 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:46.680883 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:46.680883 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:46.680883 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:46.680883 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:46.680883 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:46.680883 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:46.680883 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:46.680883 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:46.680883 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:46.681897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:46.681897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:46.681897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:46.681897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:46.681897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:46.681897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:46.681897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:46.681897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:46.681897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:46.681897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:46.681897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:46.681897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:46.681897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:46.681897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:46.681897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:46.681897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:46.681897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:46.682899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:46.682899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:46.682899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:46.682899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 496/975] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:46.688295 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903ce47-5a89-4890-941a-f3664cda6299 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:46.806648 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903ce47-5a89-4890-941a-f3664cda6299 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:46.815606 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:46.848734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac3128-4aa6-4195-b320-a0aef48e4aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:46.849465 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac3128-4aa6-4195-b320-a0aef48e4aaf None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:46.849688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac3128-4aa6-4195-b320-a0aef48e4aaf None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:46.849844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac3128-4aa6-4195-b320-a0aef48e4aaf None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:46.850597 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac3128-4aa6-4195-b320-a0aef48e4aaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:46.850918 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac3128-4aa6-4195-b320-a0aef48e4aaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:46.852366 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac3128-4aa6-4195-b320-a0aef48e4aaf None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:46.869699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 480/976] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:46.871355 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc6372-bd0b-4574-896b-76cf47c3a255 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:46.954142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:46.979108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9ccde-4421-4636-a21c-9759fed21511 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:46.979484 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9ccde-4421-4636-a21c-9759fed21511 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:46.979687 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9ccde-4421-4636-a21c-9759fed21511 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:46.981209 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 481/977] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:46.986658 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74740f-d3b6-4161-8722-4e181c9c2f20 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:46.987001 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74740f-d3b6-4161-8722-4e181c9c2f20 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:46.987203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74740f-d3b6-4161-8722-4e181c9c2f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:46.998197 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc6372-bd0b-4574-896b-76cf47c3a255 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:47.006691 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-7f74740f-d3b6-4161-8722-4e181c9c2f20 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:47.007936 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7f74740f-d3b6-4161-8722-4e181c9c2f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:47.035776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75bd95d-a17b-4969-8000-40e070616f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:47.036860 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75bd95d-a17b-4969-8000-40e070616f18 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:47.037139 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75bd95d-a17b-4969-8000-40e070616f18 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:47.037548 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75bd95d-a17b-4969-8000-40e070616f18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:47.044940 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75bd95d-a17b-4969-8000-40e070616f18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:47.046410 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75bd95d-a17b-4969-8000-40e070616f18 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:47.059434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-c75bd95d-a17b-4969-8000-40e070616f18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:47.059434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:47.059434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:47.059434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:47.059434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:47.059434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:47.059434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:47.059434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:47.059434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:47.059434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:47.059434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:47.059434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:47.059434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:38:47.059434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:38:47.059434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:38:47.060516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:38:47.060516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:47.060516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:47.060516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:47.060516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:47.060516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:47.060516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:47.060516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:47.060516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:47.060516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:47.060516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:47.060516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:47.060516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:47.060516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:47.060516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:47.060516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:47.060516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:47.061432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:38:47.061432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:38:47.061432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:38:47.061432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:38:47.061432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:38:47.061432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:38:47.061432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:47.061432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:47.061432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 497/978] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:38:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:47.066347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75bd95d-a17b-4969-8000-40e070616f18 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:47.087627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 482/979] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 08:38:47.181059 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75bd95d-a17b-4969-8000-40e070616f18 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:47.209819 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:47.220169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7086689b-a60b-4f8d-80f4-274756e7a441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:47.220816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086689b-a60b-4f8d-80f4-274756e7a441 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:47.221092 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086689b-a60b-4f8d-80f4-274756e7a441 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:47.221858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086689b-a60b-4f8d-80f4-274756e7a441 None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:47.222121 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7086689b-a60b-4f8d-80f4-274756e7a441 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:47.222425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7086689b-a60b-4f8d-80f4-274756e7a441 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:47.223953 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7086689b-a60b-4f8d-80f4-274756e7a441 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:47.243766 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 498/980] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:38:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:47.245794 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74740f-d3b6-4161-8722-4e181c9c2f20 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:47.353274 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7086689b-a60b-4f8d-80f4-274756e7a441 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:47.386562 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:47.394154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74740f-d3b6-4161-8722-4e181c9c2f20 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:47.433344 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad5664-d6ff-455e-923b-442c0e698e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:47.433871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad5664-d6ff-455e-923b-442c0e698e0c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:47.434145 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad5664-d6ff-455e-923b-442c0e698e0c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:47.434414 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad5664-d6ff-455e-923b-442c0e698e0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:47.435178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad5664-d6ff-455e-923b-442c0e698e0c None admin] RBAC: Adding query filter params (role.id=f795ff29508a4109be2917083c5ead38, scope.project.id=543d44293d894a7fbb83401f7a26dd9b, user.id=df07cffd82c74621855c8b8d7bfa989d) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:47.435482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad5664-d6ff-455e-923b-442c0e698e0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:47.437960 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad5664-d6ff-455e-923b-442c0e698e0c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:47.444361 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 483/981] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:38:47 2022] GET /identity/v3/role_assignments?user.id=df07cffd82c74621855c8b8d7bfa989d&scope.project.id=543d44293d894a7fbb83401f7a26dd9b&role.id=f795ff29508a4109be2917083c5ead38 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:47.445629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b810826-6011-4efe-a484-8bd1b8a4c566 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:47.445909 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b810826-6011-4efe-a484-8bd1b8a4c566 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:47.446201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b810826-6011-4efe-a484-8bd1b8a4c566 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:47.447900 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 484/982] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:47.454124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0153c4-bbc0-470e-bc61-fa27151dd96e None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:47.454521 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0153c4-bbc0-470e-bc61-fa27151dd96e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:47.454711 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0153c4-bbc0-470e-bc61-fa27151dd96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:47.477923 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-af0153c4-bbc0-470e-bc61-fa27151dd96e None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:47.479215 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-af0153c4-bbc0-470e-bc61-fa27151dd96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:47.483471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7086689b-a60b-4f8d-80f4-274756e7a441 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:47.539343 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d396b88-2825-4c9e-a356-fe72e3c78c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:47.540349 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d396b88-2825-4c9e-a356-fe72e3c78c24 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:47.540702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d396b88-2825-4c9e-a356-fe72e3c78c24 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:47.540971 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d396b88-2825-4c9e-a356-fe72e3c78c24 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:47.541655 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d396b88-2825-4c9e-a356-fe72e3c78c24 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:47.543066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d396b88-2825-4c9e-a356-fe72e3c78c24 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:47.561525 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 499/983] 10.222.0.71 () {60 vars in 1141 bytes} [Fri Aug 26 08:38:47 2022] POST /identity/v3/services => generated 242 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:38:47.576344 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 485/984] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:47 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:47.841686 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d396b88-2825-4c9e-a356-fe72e3c78c24 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:47.895836 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:47.959489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d396b88-2825-4c9e-a356-fe72e3c78c24 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:48.000948 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8bc99-e46f-4327-9f92-a5248f5e7826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:48.001652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8bc99-e46f-4327-9f92-a5248f5e7826 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:48.001996 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8bc99-e46f-4327-9f92-a5248f5e7826 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:48.002211 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8bc99-e46f-4327-9f92-a5248f5e7826 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:48.002777 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a8bc99-e46f-4327-9f92-a5248f5e7826 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:48.004258 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a8bc99-e46f-4327-9f92-a5248f5e7826 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:48.014094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-56a8bc99-e46f-4327-9f92-a5248f5e7826 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 08:38:48.014094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:48.014094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 08:38:48.014094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 08:38:48.014094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 08:38:48.014094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 08:38:48.014094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 08:38:48.014094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:48.014094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 08:38:48.014094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:48.014094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:38:48.014094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:48.014094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:48.014094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:48.014094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:48.014094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:48.014094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:48.015351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:48.015351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:48.015351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:48.015351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:48.015351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:48.015351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:48.015351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 08:38:48.015351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 08:38:48.015351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 08:38:48.015351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 08:38:48.015351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:48.015351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:48.015351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:48.015351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:48.015351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:48.015351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:48.016961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:48.016961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:48.016961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:48.016961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:48.016961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:48.016961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:48.016961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:48.016961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:48.016961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:48.016961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:48.016961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 08:38:48.016961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:48.016961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 08:38:48.016961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:48.016961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 500/985] 10.222.0.71 () {56 vars in 1126 bytes} [Fri Aug 26 08:38:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:48.024234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0153c4-bbc0-470e-bc61-fa27151dd96e None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:48.120854 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:48.138532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0153c4-bbc0-470e-bc61-fa27151dd96e None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:48.175571 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d157d3fa-7f0a-408f-a4c0-c3b27a3b5187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:48.176171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157d3fa-7f0a-408f-a4c0-c3b27a3b5187 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:48.176455 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157d3fa-7f0a-408f-a4c0-c3b27a3b5187 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:48.176722 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157d3fa-7f0a-408f-a4c0-c3b27a3b5187 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:48.177357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d157d3fa-7f0a-408f-a4c0-c3b27a3b5187 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:48.177666 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d157d3fa-7f0a-408f-a4c0-c3b27a3b5187 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:48.179138 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d157d3fa-7f0a-408f-a4c0-c3b27a3b5187 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:48.184601 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 486/986] 10.222.0.71 () {56 vars in 1122 bytes} [Fri Aug 26 08:38:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:48.190381 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8bc99-e46f-4327-9f92-a5248f5e7826 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:48.302222 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8bc99-e46f-4327-9f92-a5248f5e7826 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:48.318307 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:48.343082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0968a-10d3-4198-8b05-c8b54d272978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:48.343693 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0968a-10d3-4198-8b05-c8b54d272978 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:48.343930 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0968a-10d3-4198-8b05-c8b54d272978 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:48.344238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0968a-10d3-4198-8b05-c8b54d272978 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:48.344875 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0968a-10d3-4198-8b05-c8b54d272978 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:48.345159 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0968a-10d3-4198-8b05-c8b54d272978 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:48.346548 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0968a-10d3-4198-8b05-c8b54d272978 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:48.352224 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 501/987] 10.222.0.71 () {56 vars in 1122 bytes} [Fri Aug 26 08:38:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:48.359580 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d157d3fa-7f0a-408f-a4c0-c3b27a3b5187 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:48.488418 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d157d3fa-7f0a-408f-a4c0-c3b27a3b5187 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:48.527628 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0adb1-066a-4503-ab15-51b4f773174f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:48.528237 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0adb1-066a-4503-ab15-51b4f773174f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:48.528509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0adb1-066a-4503-ab15-51b4f773174f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:48.528776 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0adb1-066a-4503-ab15-51b4f773174f None admin] PATH_INFO: `/v3/endpoints` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:48.529483 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b0adb1-066a-4503-ab15-51b4f773174f None admin] RBAC: Adding query filter params (interface=admin, service_id=49514efa99d44522a84ed8bc0acb5e74, region_id=RegionOne) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:48.529786 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b0adb1-066a-4503-ab15-51b4f773174f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:48.531187 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b0adb1-066a-4503-ab15-51b4f773174f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:48.537795 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 487/988] 10.222.0.71 () {56 vars in 1249 bytes} [Fri Aug 26 08:38:48 2022] GET /identity/v3/endpoints?service_id=49514efa99d44522a84ed8bc0acb5e74&interface=admin®ion_id=RegionOne => generated 193 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:48.543599 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0968a-10d3-4198-8b05-c8b54d272978 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:48.708367 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0968a-10d3-4198-8b05-c8b54d272978 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:48.759700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12738b88-effd-4e9e-ac98-2b8fb19834f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:48.760610 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12738b88-effd-4e9e-ac98-2b8fb19834f1 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:48.761025 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12738b88-effd-4e9e-ac98-2b8fb19834f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:48.761430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12738b88-effd-4e9e-ac98-2b8fb19834f1 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:48.762417 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12738b88-effd-4e9e-ac98-2b8fb19834f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:48.764620 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12738b88-effd-4e9e-ac98-2b8fb19834f1 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:48.774923 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 502/989] 10.222.0.71 () {56 vars in 1087 bytes} [Fri Aug 26 08:38:48 2022] GET /identity/v3/services => generated 1727 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:38:49.471838 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4c315-9226-45ad-af2f-5f3283451b1d None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:49.472446 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4c315-9226-45ad-af2f-5f3283451b1d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:49.472446 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4c315-9226-45ad-af2f-5f3283451b1d None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:49.474416 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 488/990] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:49.482051 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde1ede-a9b4-49ea-a48f-0b82e1ba6096 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:49.482700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde1ede-a9b4-49ea-a48f-0b82e1ba6096 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:49.482700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde1ede-a9b4-49ea-a48f-0b82e1ba6096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:49.503732 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-1fde1ede-a9b4-49ea-a48f-0b82e1ba6096 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:49.505343 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1fde1ede-a9b4-49ea-a48f-0b82e1ba6096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:49.544116 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c371662-143f-4e76-9af8-675ea2db6897 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:49.544527 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c371662-143f-4e76-9af8-675ea2db6897 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:49.544702 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c371662-143f-4e76-9af8-675ea2db6897 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:49.546229 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 489/991] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:49.547716 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:49.553081 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ae5e5-b88a-4ef2-ad14-6e0e2a7d63de None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:49.553473 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ae5e5-b88a-4ef2-ad14-6e0e2a7d63de None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:49.553883 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ae5e5-b88a-4ef2-ad14-6e0e2a7d63de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:49.576728 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-201ae5e5-b88a-4ef2-ad14-6e0e2a7d63de None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:49.578368 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-201ae5e5-b88a-4ef2-ad14-6e0e2a7d63de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:49.583771 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 503/992] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:49 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:49.591002 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:49.658431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 490/993] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:49 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:49.851909 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde1ede-a9b4-49ea-a48f-0b82e1ba6096 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:49.934013 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ae5e5-b88a-4ef2-ad14-6e0e2a7d63de None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:49.991341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde1ede-a9b4-49ea-a48f-0b82e1ba6096 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:50.054715 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfb2db-8dbc-464f-b188-5c4e8e78903a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:50.055486 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfb2db-8dbc-464f-b188-5c4e8e78903a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:50.055768 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfb2db-8dbc-464f-b188-5c4e8e78903a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:50.056159 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfb2db-8dbc-464f-b188-5c4e8e78903a None admin] PATH_INFO: `/v3/services/image` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:50.056973 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbfb2db-8dbc-464f-b188-5c4e8e78903a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:50.059093 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbfb2db-8dbc-464f-b188-5c4e8e78903a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:50.072500 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-ecbfb2db-8dbc-464f-b188-5c4e8e78903a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 08:38:50.072500 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:50.072500 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 08:38:50.072500 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 08:38:50.072500 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 08:38:50.072500 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 08:38:50.072500 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 08:38:50.072500 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:50.072500 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 08:38:50.072500 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:50.072500 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:38:50.072500 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:50.072500 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:50.072500 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:50.072500 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:50.072500 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:50.072500 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:50.073846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:50.073846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:50.073846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:50.073846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:50.073846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:50.073846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:50.073846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 08:38:50.073846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 08:38:50.073846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 08:38:50.073846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 08:38:50.073846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:50.073846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:50.073846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:50.073846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:50.073846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:50.073846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:50.074698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:50.074698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:50.074698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:50.074698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:50.074698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:50.074698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:50.074698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:50.074698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:50.074698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:50.074698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:50.074698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 08:38:50.074698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:50.074698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 08:38:50.074698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:50.074698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 504/994] 10.222.0.71 () {56 vars in 1105 bytes} [Fri Aug 26 08:38:49 2022] GET /identity/v3/services/image => generated 86 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:38:50.080415 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfb2db-8dbc-464f-b188-5c4e8e78903a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:50.092828 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ae5e5-b88a-4ef2-ad14-6e0e2a7d63de None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:50.120910 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:50.138245 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982e2c3-d7e0-45f8-8507-71399c8b7e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:50.138467 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982e2c3-d7e0-45f8-8507-71399c8b7e8f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:50.138705 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982e2c3-d7e0-45f8-8507-71399c8b7e8f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:50.139009 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982e2c3-d7e0-45f8-8507-71399c8b7e8f None admin] PATH_INFO: `/v3/services/compute` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:50.139609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e982e2c3-d7e0-45f8-8507-71399c8b7e8f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:50.141113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e982e2c3-d7e0-45f8-8507-71399c8b7e8f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:50.151364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-e982e2c3-d7e0-45f8-8507-71399c8b7e8f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 08:38:50.151364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:50.151364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 08:38:50.151364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 08:38:50.151364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 08:38:50.151364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 08:38:50.151364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 08:38:50.151364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:50.151364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 08:38:50.151364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:50.151364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:38:50.151364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:50.151364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:50.151364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:50.151364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:50.151364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:50.151364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:50.152523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:50.152523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:50.152523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:50.152523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:50.152523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:50.152523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:50.152523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 08:38:50.152523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 08:38:50.152523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 08:38:50.152523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 08:38:50.152523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:50.152523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:50.152523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:50.152523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:50.152523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:50.152523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:50.153423 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:50.153423 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:50.153423 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:50.153423 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:50.153423 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:50.153423 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:50.153423 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:50.153423 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:50.153423 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:50.153423 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:50.153423 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 08:38:50.153423 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:50.153423 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 08:38:50.153423 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:50.153423 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 491/995] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:38:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:50.158693 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982e2c3-d7e0-45f8-8507-71399c8b7e8f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:50.173293 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:50.202641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfb2db-8dbc-464f-b188-5c4e8e78903a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:50.245169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78759cdc-c34d-42c7-91e8-b3272d012465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:50.247141 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78759cdc-c34d-42c7-91e8-b3272d012465 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:50.247426 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78759cdc-c34d-42c7-91e8-b3272d012465 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:50.247696 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78759cdc-c34d-42c7-91e8-b3272d012465 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:50.248347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78759cdc-c34d-42c7-91e8-b3272d012465 None admin] RBAC: Adding query filter params (name=image) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:50.248652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78759cdc-c34d-42c7-91e8-b3272d012465 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:50.250046 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78759cdc-c34d-42c7-91e8-b3272d012465 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:50.257099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 505/996] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:50.262689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78759cdc-c34d-42c7-91e8-b3272d012465 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:50.284438 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982e2c3-d7e0-45f8-8507-71399c8b7e8f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:50.323642 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7461184-494c-454f-ba67-e77036d6a999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:50.324351 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7461184-494c-454f-ba67-e77036d6a999 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:50.324685 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7461184-494c-454f-ba67-e77036d6a999 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:50.325019 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7461184-494c-454f-ba67-e77036d6a999 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:50.325762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7461184-494c-454f-ba67-e77036d6a999 None admin] RBAC: Adding query filter params (name=compute) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:50.326136 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7461184-494c-454f-ba67-e77036d6a999 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:50.327628 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7461184-494c-454f-ba67-e77036d6a999 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:50.334109 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 492/997] 10.222.0.71 () {56 vars in 1112 bytes} [Fri Aug 26 08:38:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:50.340161 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7461184-494c-454f-ba67-e77036d6a999 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:50.412957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:50.418108 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78759cdc-c34d-42c7-91e8-b3272d012465 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:50.436641 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:50.462297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728c87b-088e-46d2-be41-c141649aeac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:50.463292 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728c87b-088e-46d2-be41-c141649aeac8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:50.463780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728c87b-088e-46d2-be41-c141649aeac8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:50.464265 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728c87b-088e-46d2-be41-c141649aeac8 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:50.465494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7728c87b-088e-46d2-be41-c141649aeac8 None admin] RBAC: Adding query filter params (type=image) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:50.466051 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7728c87b-088e-46d2-be41-c141649aeac8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:50.468695 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7728c87b-088e-46d2-be41-c141649aeac8 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:50.474300 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7461184-494c-454f-ba67-e77036d6a999 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:50.483115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 506/998] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:50 2022] GET /identity/v3/services?type=image => generated 122 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:50.522559 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c479a24-7268-42f0-a92c-3db2fbf9f6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:50.523608 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c479a24-7268-42f0-a92c-3db2fbf9f6da None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:50.524078 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c479a24-7268-42f0-a92c-3db2fbf9f6da None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:50.524540 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c479a24-7268-42f0-a92c-3db2fbf9f6da None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:50.525388 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c479a24-7268-42f0-a92c-3db2fbf9f6da None admin] RBAC: Adding query filter params (type=compute) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:50.525867 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c479a24-7268-42f0-a92c-3db2fbf9f6da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:50.527565 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c479a24-7268-42f0-a92c-3db2fbf9f6da None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:50.534480 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 493/999] 10.222.0.71 () {56 vars in 1112 bytes} [Fri Aug 26 08:38:50 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:50.541071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728c87b-088e-46d2-be41-c141649aeac8 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:50.672280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728c87b-088e-46d2-be41-c141649aeac8 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:50.713103 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c5f05-0b83-47a9-953d-12056d61d660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:50.713641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c5f05-0b83-47a9-953d-12056d61d660 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:50.713922 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c5f05-0b83-47a9-953d-12056d61d660 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:50.714196 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c5f05-0b83-47a9-953d-12056d61d660 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:50.714952 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c5f05-0b83-47a9-953d-12056d61d660 None admin] RBAC: Adding query filter params (interface=public, service_id=a6b718994a8b43198847f2948c046c50, region_id=RegionOne) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:50.715260 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c5f05-0b83-47a9-953d-12056d61d660 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:50.716671 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c5f05-0b83-47a9-953d-12056d61d660 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:50.723013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 507/1000] 10.222.0.71 () {56 vars in 1251 bytes} [Fri Aug 26 08:38:50 2022] GET /identity/v3/endpoints?service_id=a6b718994a8b43198847f2948c046c50&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:50.726606 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a8ea7-48e7-486c-b938-ed2ce02e5e0b None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:50.726897 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a8ea7-48e7-486c-b938-ed2ce02e5e0b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:50.727193 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a8ea7-48e7-486c-b938-ed2ce02e5e0b None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:50.728876 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 494/1001] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:38:50.729468 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c5f05-0b83-47a9-953d-12056d61d660 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:50.735575 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88423af0-b439-49b5-9740-d9756bec5562 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:50.735884 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88423af0-b439-49b5-9740-d9756bec5562 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:50.736211 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88423af0-b439-49b5-9740-d9756bec5562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:50.757288 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-88423af0-b439-49b5-9740-d9756bec5562 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:50.758688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-88423af0-b439-49b5-9740-d9756bec5562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:50.770138 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:50.847383 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 495/1002] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:50 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:50.858326 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c5f05-0b83-47a9-953d-12056d61d660 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:50.895295 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600fef22-31a7-4eef-94a0-8ab70e275f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:50.895927 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600fef22-31a7-4eef-94a0-8ab70e275f69 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:50.896199 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600fef22-31a7-4eef-94a0-8ab70e275f69 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:50.896486 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600fef22-31a7-4eef-94a0-8ab70e275f69 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:50.897164 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600fef22-31a7-4eef-94a0-8ab70e275f69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:50.898600 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600fef22-31a7-4eef-94a0-8ab70e275f69 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:50.906117 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 508/1003] 10.222.0.71 () {56 vars in 1087 bytes} [Fri Aug 26 08:38:50 2022] GET /identity/v3/services => generated 1727 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 08:38:51.115598 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88423af0-b439-49b5-9740-d9756bec5562 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:51.202525 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:51.244061 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88423af0-b439-49b5-9740-d9756bec5562 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:51.282192 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c1edd-0129-4663-aae7-6fff8cf8dde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:51.282836 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c1edd-0129-4663-aae7-6fff8cf8dde8 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:51.283139 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c1edd-0129-4663-aae7-6fff8cf8dde8 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:51.283410 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c1edd-0129-4663-aae7-6fff8cf8dde8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:51.284004 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c1edd-0129-4663-aae7-6fff8cf8dde8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:51.285385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c1edd-0129-4663-aae7-6fff8cf8dde8 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:51.297387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-654c1edd-0129-4663-aae7-6fff8cf8dde8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 08:38:51.297387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:51.297387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 08:38:51.297387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 08:38:51.297387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 08:38:51.297387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 08:38:51.297387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 08:38:51.297387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:51.297387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 08:38:51.297387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:51.297387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:38:51.297387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:51.297387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:51.297387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:51.297387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:51.297387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:51.297387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:51.298581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:51.298581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:51.298581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:51.298581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:51.298581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:51.298581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:51.298581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 08:38:51.298581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 08:38:51.298581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 08:38:51.298581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 08:38:51.298581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:51.298581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:51.298581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:51.298581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:51.298581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:51.298581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:51.300231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:51.300231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:51.300231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:51.300231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:51.300231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:51.300231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:51.300231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:51.300231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:51.300231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:51.300231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:51.300231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 08:38:51.300231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:51.300231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 08:38:51.300231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:51.303000 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 496/1004] 10.222.0.71 () {56 vars in 1126 bytes} [Fri Aug 26 08:38:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:38:51.304837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600fef22-31a7-4eef-94a0-8ab70e275f69 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:51.423066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:51.434578 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600fef22-31a7-4eef-94a0-8ab70e275f69 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:51.474449 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991377a-6ed4-4c15-8a96-806c4d1201cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:51.475223 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991377a-6ed4-4c15-8a96-806c4d1201cf None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:51.475484 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991377a-6ed4-4c15-8a96-806c4d1201cf None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:51.475709 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991377a-6ed4-4c15-8a96-806c4d1201cf None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:51.476409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1991377a-6ed4-4c15-8a96-806c4d1201cf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:51.476717 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1991377a-6ed4-4c15-8a96-806c4d1201cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:51.478136 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1991377a-6ed4-4c15-8a96-806c4d1201cf None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:51.489002 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 509/1005] 10.222.0.71 () {56 vars in 1122 bytes} [Fri Aug 26 08:38:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:51.491196 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c1edd-0129-4663-aae7-6fff8cf8dde8 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:51.628893 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c1edd-0129-4663-aae7-6fff8cf8dde8 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:51.664099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:51.673000 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2d58f-f15d-4a34-a48f-e47b4e852685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:51.673648 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2d58f-f15d-4a34-a48f-e47b4e852685 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:51.674021 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2d58f-f15d-4a34-a48f-e47b4e852685 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:51.674297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2d58f-f15d-4a34-a48f-e47b4e852685 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:51.675001 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2d58f-f15d-4a34-a48f-e47b4e852685 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:51.675313 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2d58f-f15d-4a34-a48f-e47b4e852685 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:51.676760 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2d58f-f15d-4a34-a48f-e47b4e852685 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:51.687798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 497/1006] 10.222.0.71 () {56 vars in 1122 bytes} [Fri Aug 26 08:38:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:51.691486 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991377a-6ed4-4c15-8a96-806c4d1201cf None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:51.820153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991377a-6ed4-4c15-8a96-806c4d1201cf None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:51.858664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de43f2cd-9625-46bc-8e3e-83847420fbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:51.859298 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43f2cd-9625-46bc-8e3e-83847420fbbe None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:51.859573 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43f2cd-9625-46bc-8e3e-83847420fbbe None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:51.859839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43f2cd-9625-46bc-8e3e-83847420fbbe None admin] PATH_INFO: `/v3/endpoints` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:51.860533 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de43f2cd-9625-46bc-8e3e-83847420fbbe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:51.861912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de43f2cd-9625-46bc-8e3e-83847420fbbe None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:51.890913 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 510/1007] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:38:51 2022] POST /identity/v3/endpoints => generated 325 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:38:52.600351 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367a8d1-34e4-47a2-9cd5-5fb476ff3400 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:52.600939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367a8d1-34e4-47a2-9cd5-5fb476ff3400 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:52.600939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367a8d1-34e4-47a2-9cd5-5fb476ff3400 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:52.602398 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 498/1008] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:52.610117 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115a257-05dc-4636-b199-ad04487d02eb None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:52.610939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115a257-05dc-4636-b199-ad04487d02eb None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:52.610939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115a257-05dc-4636-b199-ad04487d02eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:52.635067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-1115a257-05dc-4636-b199-ad04487d02eb None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:52.636481 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1115a257-05dc-4636-b199-ad04487d02eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:52.650483 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:52.748062 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 511/1009] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:52 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:52.869803 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaad4e-69f2-4800-aba6-1c50a235995b None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:52.870149 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaad4e-69f2-4800-aba6-1c50a235995b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:52.870347 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdaad4e-69f2-4800-aba6-1c50a235995b None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:52.872327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 499/1010] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:52.877858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18606ce4-a509-48c4-8b69-419691919a86 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:52.878174 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18606ce4-a509-48c4-8b69-419691919a86 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:52.878421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18606ce4-a509-48c4-8b69-419691919a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:52.898236 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-18606ce4-a509-48c4-8b69-419691919a86 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:52.899391 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-18606ce4-a509-48c4-8b69-419691919a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:52.984449 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 512/1011] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:52 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:53.018274 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdaad4e-69f2-4800-aba6-1c50a235995b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:53.142454 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdaad4e-69f2-4800-aba6-1c50a235995b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:53.179758 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896ec2b-3ca6-413f-86a8-f2dc7e2ea9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:53.180309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896ec2b-3ca6-413f-86a8-f2dc7e2ea9c1 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:53.180586 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896ec2b-3ca6-413f-86a8-f2dc7e2ea9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:53.180940 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896ec2b-3ca6-413f-86a8-f2dc7e2ea9c1 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:53.181602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c896ec2b-3ca6-413f-86a8-f2dc7e2ea9c1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:53.183096 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c896ec2b-3ca6-413f-86a8-f2dc7e2ea9c1 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:53.201626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 500/1012] 10.222.0.71 () {60 vars in 1141 bytes} [Fri Aug 26 08:38:53 2022] POST /identity/v3/services => generated 242 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:38:53.253937 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18606ce4-a509-48c4-8b69-419691919a86 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:53.345235 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:53.397300 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18606ce4-a509-48c4-8b69-419691919a86 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:53.444112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280c732-dd3f-49c3-bd17-c7298cc5bf51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:53.444682 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280c732-dd3f-49c3-bd17-c7298cc5bf51 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:53.445087 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280c732-dd3f-49c3-bd17-c7298cc5bf51 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:53.445375 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280c732-dd3f-49c3-bd17-c7298cc5bf51 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:53.446001 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280c732-dd3f-49c3-bd17-c7298cc5bf51 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:53.448653 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280c732-dd3f-49c3-bd17-c7298cc5bf51 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:53.465588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-b280c732-dd3f-49c3-bd17-c7298cc5bf51 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 08:38:53.465588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:53.465588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 08:38:53.465588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 08:38:53.465588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 08:38:53.465588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 08:38:53.465588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 08:38:53.465588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:53.465588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 08:38:53.465588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:53.465588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:38:53.465588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:53.465588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:53.465588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:53.465588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:53.465588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:53.465588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:53.467345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:53.467345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:53.467345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:53.467345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:53.467345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:53.467345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:53.467345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 08:38:53.467345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 08:38:53.467345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 08:38:53.467345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 08:38:53.467345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:53.467345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:53.467345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:53.467345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:53.467345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:53.467345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:53.468376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:53.468376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:53.468376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:53.468376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:53.468376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:53.468376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:53.468376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:53.468376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:53.468376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:53.468376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:53.468376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 08:38:53.468376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:53.468376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 08:38:53.468376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:53.469870 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 513/1013] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:38:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:53.476302 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896ec2b-3ca6-413f-86a8-f2dc7e2ea9c1 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:53.569995 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:53.661351 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896ec2b-3ca6-413f-86a8-f2dc7e2ea9c1 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:53.722581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21e501-77b6-4e84-a67e-97a7239a7584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:53.723521 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21e501-77b6-4e84-a67e-97a7239a7584 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:53.723979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21e501-77b6-4e84-a67e-97a7239a7584 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:53.724453 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21e501-77b6-4e84-a67e-97a7239a7584 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:53.725662 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd21e501-77b6-4e84-a67e-97a7239a7584 None admin] RBAC: Adding query filter params (name=compute) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:53.726214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd21e501-77b6-4e84-a67e-97a7239a7584 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:53.728790 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd21e501-77b6-4e84-a67e-97a7239a7584 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:53.739329 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 501/1014] 10.222.0.71 () {56 vars in 1112 bytes} [Fri Aug 26 08:38:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:53.748582 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280c732-dd3f-49c3-bd17-c7298cc5bf51 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:53.865633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280c732-dd3f-49c3-bd17-c7298cc5bf51 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:53.897333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:53.904734 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd4cfe-395d-4b5b-b5ee-def504206f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:53.905373 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd4cfe-395d-4b5b-b5ee-def504206f7a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:53.905714 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd4cfe-395d-4b5b-b5ee-def504206f7a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:53.905984 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd4cfe-395d-4b5b-b5ee-def504206f7a None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:53.906672 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd4cfe-395d-4b5b-b5ee-def504206f7a None admin] RBAC: Adding query filter params (type=compute) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:53.907023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd4cfe-395d-4b5b-b5ee-def504206f7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:53.908424 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd4cfe-395d-4b5b-b5ee-def504206f7a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:53.919082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 514/1015] 10.222.0.71 () {56 vars in 1112 bytes} [Fri Aug 26 08:38:53 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:53.920877 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21e501-77b6-4e84-a67e-97a7239a7584 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:53.947315 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2fa5a-b896-4856-b924-3d946361eba6 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:53.947599 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2fa5a-b896-4856-b924-3d946361eba6 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:53.947881 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2fa5a-b896-4856-b924-3d946361eba6 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:53.949301 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 515/1016] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:53.955593 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cc956-e11f-46c7-89f8-b0e6ed8bdd10 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:53.955933 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cc956-e11f-46c7-89f8-b0e6ed8bdd10 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:53.956139 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cc956-e11f-46c7-89f8-b0e6ed8bdd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:53.980739 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e49cc956-e11f-46c7-89f8-b0e6ed8bdd10 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:53.982538 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e49cc956-e11f-46c7-89f8-b0e6ed8bdd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:54.040208 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21e501-77b6-4e84-a67e-97a7239a7584 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:54.062667 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:54.082426 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 516/1017] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:53 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:54.082652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c34fb-56d4-4520-8bbc-64ea0956de57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:54.083288 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c34fb-56d4-4520-8bbc-64ea0956de57 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:54.083617 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c34fb-56d4-4520-8bbc-64ea0956de57 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:54.084021 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c34fb-56d4-4520-8bbc-64ea0956de57 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:54.085116 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c34fb-56d4-4520-8bbc-64ea0956de57 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:54.086638 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c34fb-56d4-4520-8bbc-64ea0956de57 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:54.112354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 502/1018] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:38:53 2022] POST /identity/v3/endpoints => generated 334 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:38:54.360265 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49cc956-e11f-46c7-89f8-b0e6ed8bdd10 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:54.498349 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49cc956-e11f-46c7-89f8-b0e6ed8bdd10 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:54.536199 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:54.562308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba757432-07bd-4e10-95ae-23e5200ede49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:54.563426 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba757432-07bd-4e10-95ae-23e5200ede49 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:54.563869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba757432-07bd-4e10-95ae-23e5200ede49 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:54.564334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba757432-07bd-4e10-95ae-23e5200ede49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:54.566114 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba757432-07bd-4e10-95ae-23e5200ede49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:54.568827 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba757432-07bd-4e10-95ae-23e5200ede49 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:54.572123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 517/1019] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:54.578875 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c34fb-56d4-4520-8bbc-64ea0956de57 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:54.701500 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c34fb-56d4-4520-8bbc-64ea0956de57 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:54.738959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1b180-0089-41a5-ac48-9c37ebab685d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:54.739544 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1b180-0089-41a5-ac48-9c37ebab685d None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:54.739820 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1b180-0089-41a5-ac48-9c37ebab685d None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:54.740254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1b180-0089-41a5-ac48-9c37ebab685d None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:54.741218 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa1b180-0089-41a5-ac48-9c37ebab685d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:54.742661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa1b180-0089-41a5-ac48-9c37ebab685d None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:54.776605 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 503/1020] 10.222.0.71 () {60 vars in 1141 bytes} [Fri Aug 26 08:38:54 2022] POST /identity/v3/projects => generated 311 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:38:55.312674 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d329e-7c90-417f-a6f8-39df20ccfb7b None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:55.313423 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d329e-7c90-417f-a6f8-39df20ccfb7b None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:55.313524 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d329e-7c90-417f-a6f8-39df20ccfb7b None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:55.315352 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 518/1021] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:55.323654 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf94e0-0fec-40ba-9dfb-82a70174ee32 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:55.324137 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf94e0-0fec-40ba-9dfb-82a70174ee32 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:55.324607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf94e0-0fec-40ba-9dfb-82a70174ee32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:55.359850 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-f5cf94e0-0fec-40ba-9dfb-82a70174ee32 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:55.361334 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f5cf94e0-0fec-40ba-9dfb-82a70174ee32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:55.398128 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:55.478926 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 504/1022] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:55.757810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d329e-7c90-417f-a6f8-39df20ccfb7b None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:55.890642 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d329e-7c90-417f-a6f8-39df20ccfb7b None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:55.930358 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9aad19-e0cd-4b9b-98a4-858da45fe424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:55.931228 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9aad19-e0cd-4b9b-98a4-858da45fe424 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:55.931405 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9aad19-e0cd-4b9b-98a4-858da45fe424 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:55.931656 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9aad19-e0cd-4b9b-98a4-858da45fe424 None admin] PATH_INFO: `/v3/services/image` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:55.932155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9aad19-e0cd-4b9b-98a4-858da45fe424 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:55.933527 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9aad19-e0cd-4b9b-98a4-858da45fe424 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:55.942928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-2d9aad19-e0cd-4b9b-98a4-858da45fe424 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 08:38:55.942928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:55.942928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 08:38:55.942928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 08:38:55.942928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 08:38:55.942928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 08:38:55.942928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 08:38:55.942928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:55.942928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 08:38:55.942928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:55.942928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:38:55.942928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:55.942928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:55.942928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:55.942928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:55.942928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:55.942928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:55.944069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:55.944069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:55.944069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:55.944069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:55.944069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:55.944069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:55.944069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 08:38:55.944069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 08:38:55.944069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 08:38:55.944069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 08:38:55.944069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:55.944069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:55.944069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:55.944069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:55.944069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:55.944069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:55.945719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:55.945719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:55.945719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:55.945719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:55.945719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:55.945719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:55.945719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:55.945719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:55.945719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:55.945719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:55.945719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 08:38:55.945719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:55.945719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 08:38:55.945719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:55.945719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 519/1023] 10.222.0.71 () {56 vars in 1105 bytes} [Fri Aug 26 08:38:55 2022] GET /identity/v3/services/image => generated 86 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:55.950326 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf94e0-0fec-40ba-9dfb-82a70174ee32 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:56.076024 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf94e0-0fec-40ba-9dfb-82a70174ee32 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:56.127929 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d5b9f-599a-4fec-91b2-bd45082f1cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:56.128558 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d5b9f-599a-4fec-91b2-bd45082f1cff None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:56.128813 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d5b9f-599a-4fec-91b2-bd45082f1cff None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:56.129082 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d5b9f-599a-4fec-91b2-bd45082f1cff None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:56.129719 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d5b9f-599a-4fec-91b2-bd45082f1cff None admin] RBAC: Adding query filter params (name=image) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:56.130030 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d5b9f-599a-4fec-91b2-bd45082f1cff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:56.131725 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d5b9f-599a-4fec-91b2-bd45082f1cff None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:56.137282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 505/1024] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:56.142981 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9aad19-e0cd-4b9b-98a4-858da45fe424 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:56.203849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:56.231017 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf271d4-26ec-4885-b988-d95bf2a2fc27 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:56.231382 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf271d4-26ec-4885-b988-d95bf2a2fc27 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:56.231562 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf271d4-26ec-4885-b988-d95bf2a2fc27 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:56.236643 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 506/1025] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:56.241979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eea7d8-9e91-4d35-8b1e-e23f8d528702 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:56.242298 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eea7d8-9e91-4d35-8b1e-e23f8d528702 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:56.242572 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eea7d8-9e91-4d35-8b1e-e23f8d528702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:56.263140 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-a7eea7d8-9e91-4d35-8b1e-e23f8d528702 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:56.264549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a7eea7d8-9e91-4d35-8b1e-e23f8d528702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:56.280603 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9aad19-e0cd-4b9b-98a4-858da45fe424 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:56.281267 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:56.332434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913b358-a78c-40c2-a904-8b004b6829cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:56.333700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913b358-a78c-40c2-a904-8b004b6829cb None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:56.333987 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913b358-a78c-40c2-a904-8b004b6829cb None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:56.334331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913b358-a78c-40c2-a904-8b004b6829cb None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:56.335162 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8913b358-a78c-40c2-a904-8b004b6829cb None admin] RBAC: Adding query filter params (type=image) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:56.335474 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8913b358-a78c-40c2-a904-8b004b6829cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:56.336890 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8913b358-a78c-40c2-a904-8b004b6829cb None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:56.342475 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 520/1026] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:56.350664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913b358-a78c-40c2-a904-8b004b6829cb None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:56.354733 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 507/1027] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:38:56.493126 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913b358-a78c-40c2-a904-8b004b6829cb None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:56.515086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:56.536116 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b12034-aab4-4136-8690-6e131c9c8cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:56.536718 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b12034-aab4-4136-8690-6e131c9c8cd8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:56.537010 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b12034-aab4-4136-8690-6e131c9c8cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:56.537304 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b12034-aab4-4136-8690-6e131c9c8cd8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:56.538087 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b12034-aab4-4136-8690-6e131c9c8cd8 None admin] RBAC: Adding query filter params (interface=public, service_id=07b2f1caa28447b4bf818a5bc5264da1, region_id=RegionOne) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:56.538393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b12034-aab4-4136-8690-6e131c9c8cd8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:56.539813 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b12034-aab4-4136-8690-6e131c9c8cd8 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:56.546107 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 521/1028] 10.222.0.71 () {56 vars in 1251 bytes} [Fri Aug 26 08:38:56 2022] GET /identity/v3/endpoints?service_id=07b2f1caa28447b4bf818a5bc5264da1&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:56.551758 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eea7d8-9e91-4d35-8b1e-e23f8d528702 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:56.625616 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b12034-aab4-4136-8690-6e131c9c8cd8 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:56.670459 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eea7d8-9e91-4d35-8b1e-e23f8d528702 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:56.721364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb4148-f8c1-4219-9a8c-da77320b5b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:56.722438 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb4148-f8c1-4219-9a8c-da77320b5b81 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:56.722967 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb4148-f8c1-4219-9a8c-da77320b5b81 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:56.723458 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb4148-f8c1-4219-9a8c-da77320b5b81 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:56.724290 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb4148-f8c1-4219-9a8c-da77320b5b81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:56.726374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb4148-f8c1-4219-9a8c-da77320b5b81 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:56.734779 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 508/1029] 10.222.0.71 () {56 vars in 1087 bytes} [Fri Aug 26 08:38:56 2022] GET /identity/v3/services => generated 1957 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:38:56.778195 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b12034-aab4-4136-8690-6e131c9c8cd8 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:56.824812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca79633-9b8c-432a-9516-c6e580c98dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:56.825486 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca79633-9b8c-432a-9516-c6e580c98dc8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:56.825756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca79633-9b8c-432a-9516-c6e580c98dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:56.826034 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca79633-9b8c-432a-9516-c6e580c98dc8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:56.841223 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca79633-9b8c-432a-9516-c6e580c98dc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:38:56.846172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-1ca79633-9b8c-432a-9516-c6e580c98dc8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 08:38:56.846172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:56.846172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:56.846172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:56.846172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:56.846172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:56.846172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:56.846172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:56.846172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:56.846172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:56.846172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:56.846172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:56.846172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:38:56.846172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:38:56.846172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:38:56.846172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:38:56.847254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:38:56.847254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:38:56.847254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:38:56.847254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:38:56.847254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:56.847254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:56.847254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:56.847254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:56.847254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:56.847254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:56.847254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:56.847254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:56.847254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:56.847254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:56.847254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:56.847254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:56.848118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:56.848118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:56.848118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:56.848118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:56.848118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:38:56.848118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:38:56.848118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:38:56.848118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:38:56.848118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:38:56.848118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:38:56.848118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 08:38:56.848118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:56.848118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 522/1030] 10.222.0.71 () {56 vars in 1120 bytes} [Fri Aug 26 08:38:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:38:56.856538 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb4148-f8c1-4219-9a8c-da77320b5b81 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:56.889708 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:56.952171 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903821f-18b6-475a-9725-ac7bf35b8fb2 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:56.952494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903821f-18b6-475a-9725-ac7bf35b8fb2 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:56.952686 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903821f-18b6-475a-9725-ac7bf35b8fb2 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:56.954207 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 523/1031] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:56.960123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3d0d1-a4a7-4905-b563-cb11d44dd538 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:56.960123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3d0d1-a4a7-4905-b563-cb11d44dd538 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:56.960416 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3d0d1-a4a7-4905-b563-cb11d44dd538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:56.982700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-0be3d0d1-a4a7-4905-b563-cb11d44dd538 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:56.983954 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-0be3d0d1-a4a7-4905-b563-cb11d44dd538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:57.031378 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:57.031920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb4148-f8c1-4219-9a8c-da77320b5b81 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:57.076007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c5a65-b41b-494f-8cd3-a32961609f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:57.076765 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c5a65-b41b-494f-8cd3-a32961609f77 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:57.077107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c5a65-b41b-494f-8cd3-a32961609f77 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:57.077552 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c5a65-b41b-494f-8cd3-a32961609f77 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:57.078575 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c5a65-b41b-494f-8cd3-a32961609f77 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:57.079094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c5a65-b41b-494f-8cd3-a32961609f77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:57.081438 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c5a65-b41b-494f-8cd3-a32961609f77 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:57.095284 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 524/1032] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 08:38:57.096860 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 509/1033] 10.222.0.71 () {56 vars in 1115 bytes} [Fri Aug 26 08:38:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:57.102975 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3d0d1-a4a7-4905-b563-cb11d44dd538 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:57.227535 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3d0d1-a4a7-4905-b563-cb11d44dd538 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:57.292937 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205f218-4423-4ab2-9f5c-e284728f0695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:57.293600 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205f218-4423-4ab2-9f5c-e284728f0695 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:57.293905 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205f218-4423-4ab2-9f5c-e284728f0695 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:57.294178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205f218-4423-4ab2-9f5c-e284728f0695 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:57.308882 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e205f218-4423-4ab2-9f5c-e284728f0695 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:57.310413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e205f218-4423-4ab2-9f5c-e284728f0695 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:57.343414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-e205f218-4423-4ab2-9f5c-e284728f0695 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:57.343414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:57.343414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:57.343414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:57.343414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:57.343414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:57.343414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:57.343414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:57.343414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:57.343414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:57.343414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:57.343414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:57.343414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:57.343414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:57.343414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:57.343414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:57.345514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:57.345514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:57.345514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:57.345514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:57.345514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:57.345514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:57.345514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:57.345514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:57.345514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:57.345514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:57.345514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:57.345514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:57.345514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:57.345514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:57.345514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:57.345514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:57.345514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:57.346571 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:57.346571 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:57.346571 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:57.351365 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 525/1034] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:57.357103 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c5a65-b41b-494f-8cd3-a32961609f77 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:57.402267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205f218-4423-4ab2-9f5c-e284728f0695 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:57.438360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:57.497391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:57.522784 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c5a65-b41b-494f-8cd3-a32961609f77 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:57.540532 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205f218-4423-4ab2-9f5c-e284728f0695 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:57.568867 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08548a5f-52c9-4589-9ad7-dd41a820bc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:57.569629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08548a5f-52c9-4589-9ad7-dd41a820bc9b None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:57.569902 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08548a5f-52c9-4589-9ad7-dd41a820bc9b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:57.570167 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08548a5f-52c9-4589-9ad7-dd41a820bc9b None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:57.570818 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08548a5f-52c9-4589-9ad7-dd41a820bc9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:57.571145 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08548a5f-52c9-4589-9ad7-dd41a820bc9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:57.572631 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08548a5f-52c9-4589-9ad7-dd41a820bc9b None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:57.589928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b43b0-731f-4874-91cb-f7f50f857834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:57.590908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b43b0-731f-4874-91cb-f7f50f857834 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:57.591334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b43b0-731f-4874-91cb-f7f50f857834 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:57.591792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b43b0-731f-4874-91cb-f7f50f857834 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:57.592577 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 510/1035] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:57.597913 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b43b0-731f-4874-91cb-f7f50f857834 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:57.598270 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08548a5f-52c9-4589-9ad7-dd41a820bc9b None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:57.599408 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b43b0-731f-4874-91cb-f7f50f857834 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:57.601491 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 526/1036] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:57.607086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b43b0-731f-4874-91cb-f7f50f857834 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:57.722301 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:57.728610 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08548a5f-52c9-4589-9ad7-dd41a820bc9b None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:57.735046 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b43b0-731f-4874-91cb-f7f50f857834 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:57.781606 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc21d90-9dd7-4c30-9556-0e7f24c187a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:57.782687 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc21d90-9dd7-4c30-9556-0e7f24c187a2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:57.783172 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc21d90-9dd7-4c30-9556-0e7f24c187a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:57.783722 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc21d90-9dd7-4c30-9556-0e7f24c187a2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:57.784516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376e625-e7a2-4b00-9c9b-4763071d1a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:57.785516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376e625-e7a2-4b00-9c9b-4763071d1a47 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:57.785974 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376e625-e7a2-4b00-9c9b-4763071d1a47 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:57.786424 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376e625-e7a2-4b00-9c9b-4763071d1a47 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:57.787531 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9376e625-e7a2-4b00-9c9b-4763071d1a47 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:57.788313 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:57.789887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9376e625-e7a2-4b00-9c9b-4763071d1a47 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:57.797145 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc21d90-9dd7-4c30-9556-0e7f24c187a2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:57.799048 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc21d90-9dd7-4c30-9556-0e7f24c187a2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:57.815532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-cdc21d90-9dd7-4c30-9556-0e7f24c187a2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 08:38:57.815532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:57.815532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:57.815532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:57.815532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:57.815532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:57.815532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:57.815532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:57.815532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:57.815532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:57.815532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:57.815532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:57.815532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:38:57.815532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:38:57.815532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:38:57.815532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:38:57.816924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:57.816924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:57.816924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:38:57.816924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:57.816924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:38:57.816924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:38:57.816924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:57.816924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:57.816924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:57.816924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:57.816924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:57.816924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:57.816924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:57.816924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:57.816924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:57.816924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:57.816924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:57.816924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:57.819169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:57.819169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:57.819169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:38:57.819169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:38:57.819169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:38:57.819169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:38:57.819169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:38:57.819169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:38:57.819169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 08:38:57.819169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:57.819169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 511/1037] 10.222.0.71 () {56 vars in 1093 bytes} [Fri Aug 26 08:38:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:38:57.824472 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc21d90-9dd7-4c30-9556-0e7f24c187a2 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:57.827120 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 527/1038] 10.222.0.71 () {60 vars in 1132 bytes} [Fri Aug 26 08:38:57 2022] POST /identity/v3/users => generated 285 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:38:57.968115 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc21d90-9dd7-4c30-9556-0e7f24c187a2 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:58.010251 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da3a7c-4829-4a14-ac77-a1c164b6cc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:58.010864 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da3a7c-4829-4a14-ac77-a1c164b6cc74 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:58.011164 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da3a7c-4829-4a14-ac77-a1c164b6cc74 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:58.011431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da3a7c-4829-4a14-ac77-a1c164b6cc74 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:58.012166 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da3a7c-4829-4a14-ac77-a1c164b6cc74 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:58.012469 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da3a7c-4829-4a14-ac77-a1c164b6cc74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:58.014222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da3a7c-4829-4a14-ac77-a1c164b6cc74 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:58.039085 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 512/1039] 10.222.0.71 () {56 vars in 1133 bytes} [Fri Aug 26 08:38:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:58.045377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376e625-e7a2-4b00-9c9b-4763071d1a47 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:58.174778 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376e625-e7a2-4b00-9c9b-4763071d1a47 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:58.211634 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d672d3-ffb4-4d80-a3b0-aad2c3456140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:58.212255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d672d3-ffb4-4d80-a3b0-aad2c3456140 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:58.212534 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d672d3-ffb4-4d80-a3b0-aad2c3456140 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:58.212803 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d672d3-ffb4-4d80-a3b0-aad2c3456140 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:58.230581 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:58.288623 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d672d3-ffb4-4d80-a3b0-aad2c3456140 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:58.290735 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d672d3-ffb4-4d80-a3b0-aad2c3456140 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:58.428809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 528/1040] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:38:58 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 08:38:58.435271 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da3a7c-4829-4a14-ac77-a1c164b6cc74 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:58.506345 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:58.573771 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da3a7c-4829-4a14-ac77-a1c164b6cc74 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:58.623991 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86f169-ea50-475a-ba83-dc0024e715a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:58.624787 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86f169-ea50-475a-ba83-dc0024e715a2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:58.625067 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86f169-ea50-475a-ba83-dc0024e715a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:58.625365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86f169-ea50-475a-ba83-dc0024e715a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:58.637222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86f169-ea50-475a-ba83-dc0024e715a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:58.638679 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86f169-ea50-475a-ba83-dc0024e715a2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:58.660357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-7b86f169-ea50-475a-ba83-dc0024e715a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:58.660357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:58.660357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:58.660357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:58.660357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:58.660357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:58.660357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:58.660357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:58.660357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:58.660357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:58.660357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:58.660357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:58.660357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:38:58.660357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:38:58.660357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:38:58.660357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:38:58.661832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:58.661832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:58.661832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:58.661832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:58.661832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:58.661832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:58.661832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:58.661832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:58.661832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:58.661832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:58.661832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:58.661832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:58.661832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:58.661832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:58.661832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:58.661832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:58.661832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:38:58.662734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:38:58.662734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:38:58.662734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:38:58.662734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 513/1041] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:58.663142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d672d3-ffb4-4d80-a3b0-aad2c3456140 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:58.716933 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:58.754515 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7780d-fdb5-4bfc-9493-a7b8007000d6 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:58.754737 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7780d-fdb5-4bfc-9493-a7b8007000d6 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:58.755027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7780d-fdb5-4bfc-9493-a7b8007000d6 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:58.756707 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 514/1042] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:38:58.761965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6cba6f-349b-4537-bf3c-4c0ac362f056 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:58.762224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6cba6f-349b-4537-bf3c-4c0ac362f056 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:58.762495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6cba6f-349b-4537-bf3c-4c0ac362f056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:58.781559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d672d3-ffb4-4d80-a3b0-aad2c3456140 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:58.781806 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-1a6cba6f-349b-4537-bf3c-4c0ac362f056 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:38:58.782744 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1a6cba6f-349b-4537-bf3c-4c0ac362f056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:38:58.819185 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44419b9-bfe5-4a79-9411-f287a4a62411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:58.820116 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44419b9-bfe5-4a79-9411-f287a4a62411 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:58.820479 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44419b9-bfe5-4a79-9411-f287a4a62411 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:58.820750 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44419b9-bfe5-4a79-9411-f287a4a62411 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:58.821399 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44419b9-bfe5-4a79-9411-f287a4a62411 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:58.821699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44419b9-bfe5-4a79-9411-f287a4a62411 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:58.823164 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44419b9-bfe5-4a79-9411-f287a4a62411 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:58.837227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 529/1043] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:58.842703 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44419b9-bfe5-4a79-9411-f287a4a62411 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:58.863094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 515/1044] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 08:38:58.962640 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44419b9-bfe5-4a79-9411-f287a4a62411 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:58.971924 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:59.005553 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02693f1c-277a-4611-b7c7-da865155e77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:59.006243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02693f1c-277a-4611-b7c7-da865155e77d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:59.006595 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02693f1c-277a-4611-b7c7-da865155e77d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:59.007029 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02693f1c-277a-4611-b7c7-da865155e77d None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:59.015869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02693f1c-277a-4611-b7c7-da865155e77d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:59.018503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02693f1c-277a-4611-b7c7-da865155e77d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:59.033058 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-02693f1c-277a-4611-b7c7-da865155e77d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:59.033058 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:59.033058 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:59.033058 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:59.033058 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:59.033058 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:59.033058 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:59.033058 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:59.033058 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:59.033058 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:59.033058 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:59.033058 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:59.033058 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:38:59.033058 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:38:59.033058 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:38:59.034285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:38:59.034285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:59.034285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:59.034285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:59.034285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:59.034285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:59.034285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:59.034285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:59.034285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:59.034285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:59.034285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:59.034285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:59.034285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:59.034285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:59.034285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:59.034285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:59.034285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:59.035569 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:38:59.035569 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:38:59.035569 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:38:59.035569 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:38:59.035569 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:38:59.035569 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:38:59.035569 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:38:59.035569 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:59.039310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 530/1045] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:38:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:38:59.041096 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6cba6f-349b-4537-bf3c-4c0ac362f056 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:59.079719 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:59.127299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02693f1c-277a-4611-b7c7-da865155e77d None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:59.157359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6cba6f-349b-4537-bf3c-4c0ac362f056 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:59.199933 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc6d3c7-24b6-4dd7-9836-eea11e82b277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:59.200735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6d3c7-24b6-4dd7-9836-eea11e82b277 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:59.201105 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6d3c7-24b6-4dd7-9836-eea11e82b277 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:59.201467 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc6d3c7-24b6-4dd7-9836-eea11e82b277 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:59.202538 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc6d3c7-24b6-4dd7-9836-eea11e82b277 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:59.202538 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc6d3c7-24b6-4dd7-9836-eea11e82b277 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:59.204154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc6d3c7-24b6-4dd7-9836-eea11e82b277 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:59.217200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 516/1046] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:38:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:59.223220 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc6d3c7-24b6-4dd7-9836-eea11e82b277 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:59.262360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02693f1c-277a-4611-b7c7-da865155e77d None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:59.296472 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a23c2-7b58-46fd-9d3d-f0b2ab98c972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:59.297108 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a23c2-7b58-46fd-9d3d-f0b2ab98c972 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:59.297391 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a23c2-7b58-46fd-9d3d-f0b2ab98c972 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:59.297665 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a23c2-7b58-46fd-9d3d-f0b2ab98c972 None admin] PATH_INFO: `/v3/services/image` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:59.298276 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03a23c2-7b58-46fd-9d3d-f0b2ab98c972 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:59.299700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03a23c2-7b58-46fd-9d3d-f0b2ab98c972 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:59.308722 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-a03a23c2-7b58-46fd-9d3d-f0b2ab98c972 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 08:38:59.308722 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:59.308722 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 08:38:59.308722 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 08:38:59.308722 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 08:38:59.308722 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 08:38:59.308722 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 08:38:59.308722 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:59.308722 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 08:38:59.308722 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:59.308722 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:38:59.308722 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:59.308722 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:38:59.308722 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:38:59.308722 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:38:59.308722 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:38:59.308722 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:38:59.309780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:38:59.309780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:38:59.309780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:38:59.309780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:38:59.309780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:38:59.309780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:38:59.309780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 08:38:59.309780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 08:38:59.309780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 08:38:59.309780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 08:38:59.309780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:38:59.309780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:38:59.309780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:38:59.309780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:38:59.309780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:38:59.309780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:38:59.312155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:38:59.312155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:38:59.312155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:38:59.312155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:38:59.312155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:38:59.312155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:38:59.312155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:38:59.312155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:38:59.312155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:38:59.312155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:38:59.312155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 08:38:59.312155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:38:59.312155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 08:38:59.312155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:38:59.312155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 531/1047] 10.222.0.71 () {56 vars in 1105 bytes} [Fri Aug 26 08:38:59 2022] GET /identity/v3/services/image => generated 86 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:38:59.315728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a23c2-7b58-46fd-9d3d-f0b2ab98c972 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:59.332804 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:59.341046 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc6d3c7-24b6-4dd7-9836-eea11e82b277 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:59.359687 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:59.384023 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb865ae-a41f-4e16-93ca-04f6965524cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:59.384633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb865ae-a41f-4e16-93ca-04f6965524cd None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:59.384910 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb865ae-a41f-4e16-93ca-04f6965524cd None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:59.385326 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb865ae-a41f-4e16-93ca-04f6965524cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:59.386276 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb865ae-a41f-4e16-93ca-04f6965524cd None admin] RBAC: Adding query filter params (role.id=f795ff29508a4109be2917083c5ead38, scope.project.id=543d44293d894a7fbb83401f7a26dd9b, user.id=f16d1768715a41648af3fe6afadd7477) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:59.386276 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb865ae-a41f-4e16-93ca-04f6965524cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:59.387696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb865ae-a41f-4e16-93ca-04f6965524cd None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:59.402922 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 517/1048] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:38:59 2022] GET /identity/v3/role_assignments?user.id=f16d1768715a41648af3fe6afadd7477&scope.project.id=543d44293d894a7fbb83401f7a26dd9b&role.id=f795ff29508a4109be2917083c5ead38 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:59.452253 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a23c2-7b58-46fd-9d3d-f0b2ab98c972 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:59.501942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ebe78b-064f-4662-9612-b3441cfd5429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:59.502728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebe78b-064f-4662-9612-b3441cfd5429 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:59.503146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebe78b-064f-4662-9612-b3441cfd5429 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:59.503527 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebe78b-064f-4662-9612-b3441cfd5429 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:59.504276 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ebe78b-064f-4662-9612-b3441cfd5429 None admin] RBAC: Adding query filter params (name=image) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:59.504671 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ebe78b-064f-4662-9612-b3441cfd5429 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:59.506225 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ebe78b-064f-4662-9612-b3441cfd5429 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:59.512387 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 532/1049] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:38:59.518618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb865ae-a41f-4e16-93ca-04f6965524cd None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:59.636659 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb865ae-a41f-4e16-93ca-04f6965524cd None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:59.675694 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad353a3-1fea-4073-b371-ae5090107a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:59.676743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad353a3-1fea-4073-b371-ae5090107a8b None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:59.676743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad353a3-1fea-4073-b371-ae5090107a8b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:59.676743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad353a3-1fea-4073-b371-ae5090107a8b None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:59.677477 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad353a3-1fea-4073-b371-ae5090107a8b None admin] RBAC: Adding query filter params (type=image) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:38:59.677678 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad353a3-1fea-4073-b371-ae5090107a8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:59.679148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad353a3-1fea-4073-b371-ae5090107a8b None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:59.685480 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 518/1050] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:38:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:38:59.691374 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ebe78b-064f-4662-9612-b3441cfd5429 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:38:59.809200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:38:59.817811 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ebe78b-064f-4662-9612-b3441cfd5429 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:38:59.857022 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc160c-ba24-4f0c-8f14-d14007d8de4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:38:59.857634 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc160c-ba24-4f0c-8f14-d14007d8de4f None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:59.857909 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc160c-ba24-4f0c-8f14-d14007d8de4f None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:59.858202 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc160c-ba24-4f0c-8f14-d14007d8de4f None admin] PATH_INFO: `/v3/endpoints` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:59.858871 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc160c-ba24-4f0c-8f14-d14007d8de4f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:38:59.860264 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc160c-ba24-4f0c-8f14-d14007d8de4f None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:38:59.895070 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 533/1051] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:38:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:38:59.984409 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109bb04-edbe-4fcf-8809-bc970934803a None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:59.984786 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109bb04-edbe-4fcf-8809-bc970934803a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:59.984913 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2109bb04-edbe-4fcf-8809-bc970934803a None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:38:59.986506 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 519/1052] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:38:59.992391 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3490f-9b7b-4e3c-9acf-6bb4b39deeb9 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:38:59.992653 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3490f-9b7b-4e3c-9acf-6bb4b39deeb9 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:38:59.992941 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3490f-9b7b-4e3c-9acf-6bb4b39deeb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:00.021322 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-1ee3490f-9b7b-4e3c-9acf-6bb4b39deeb9 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:39:00.023563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1ee3490f-9b7b-4e3c-9acf-6bb4b39deeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:39:00.124959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 534/1053] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:38:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:39:00.389268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109bb04-edbe-4fcf-8809-bc970934803a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:00.426521 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:00.520117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2109bb04-edbe-4fcf-8809-bc970934803a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:00.560591 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8a79a-c045-40e9-9553-1850f112bbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:00.561250 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8a79a-c045-40e9-9553-1850f112bbd4 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:00.561527 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8a79a-c045-40e9-9553-1850f112bbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:00.561884 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8a79a-c045-40e9-9553-1850f112bbd4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:00.574241 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8a79a-c045-40e9-9553-1850f112bbd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:39:00.579331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-51a8a79a-c045-40e9-9553-1850f112bbd4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:39:00.579331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:39:00.579331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:39:00.579331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:39:00.579331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:39:00.579331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:39:00.579331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:39:00.579331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:39:00.579331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:39:00.579331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:39:00.579331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:39:00.579331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:39:00.579331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:39:00.579331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:39:00.579331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:39:00.579331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:39:00.580615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:39:00.580615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:39:00.580615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:39:00.580615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:39:00.580615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:39:00.580615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:39:00.580615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:39:00.580615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:39:00.580615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:39:00.580615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:39:00.580615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:39:00.580615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:39:00.580615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:39:00.580615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:39:00.580615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:39:00.580615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:39:00.581615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:39:00.581615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:39:00.581615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:39:00.581615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:39:00.581615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:39:00.581615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:39:00.581615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:39:00.581615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:39:00.581615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:39:00.581615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:39:00.581615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:39:00.581615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:39:00.585958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 520/1054] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:39:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:39:00.586849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3490f-9b7b-4e3c-9acf-6bb4b39deeb9 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:00.670938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:00.699239 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3490f-9b7b-4e3c-9acf-6bb4b39deeb9 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:00.736341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c8df4-61cc-4bab-be44-2b7a582e8e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:00.736972 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c8df4-61cc-4bab-be44-2b7a582e8e20 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:00.737249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c8df4-61cc-4bab-be44-2b7a582e8e20 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:00.737519 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c8df4-61cc-4bab-be44-2b7a582e8e20 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:00.738254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c8df4-61cc-4bab-be44-2b7a582e8e20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:00.738578 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c8df4-61cc-4bab-be44-2b7a582e8e20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:00.740124 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2c8df4-61cc-4bab-be44-2b7a582e8e20 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:00.759862 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 535/1055] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:39:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:39:00.762799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8a79a-c045-40e9-9553-1850f112bbd4 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:00.892084 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:00.914196 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8a79a-c045-40e9-9553-1850f112bbd4 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:00.956075 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3e6f0-7fcd-4876-bf56-06505c757fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:00.956994 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3e6f0-7fcd-4876-bf56-06505c757fef None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:00.956994 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3e6f0-7fcd-4876-bf56-06505c757fef None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:00.957183 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3e6f0-7fcd-4876-bf56-06505c757fef None admin] PATH_INFO: `/v3/users/de5add03e9eb46e1ba04ce0121d21dd7` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:00.972926 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3e6f0-7fcd-4876-bf56-06505c757fef None admin] RBAC: Authorizing `identity:get_user(user_id=de5add03e9eb46e1ba04ce0121d21dd7)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:00.974395 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3e6f0-7fcd-4876-bf56-06505c757fef None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:00.976392 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 521/1056] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:39:00 2022] GET /identity/v3/users/de5add03e9eb46e1ba04ce0121d21dd7 => generated 285 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:00.982293 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c8df4-61cc-4bab-be44-2b7a582e8e20 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:01.098433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c8df4-61cc-4bab-be44-2b7a582e8e20 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:01.139042 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d3245-14d5-4093-a389-b3244a87981f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:01.139648 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d3245-14d5-4093-a389-b3244a87981f None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:01.139926 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d3245-14d5-4093-a389-b3244a87981f None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:01.140200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d3245-14d5-4093-a389-b3244a87981f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:01.151166 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:01.226324 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d3245-14d5-4093-a389-b3244a87981f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:01.227819 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d3245-14d5-4093-a389-b3244a87981f None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:01.319697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6469bf9-89c0-4fe7-9ad3-07d29fd438ef None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:01.320002 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6469bf9-89c0-4fe7-9ad3-07d29fd438ef None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:01.320408 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6469bf9-89c0-4fe7-9ad3-07d29fd438ef None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:01.321950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 522/1057] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:39:01.327896 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838c85b-df0b-4d59-8323-ab9d9351c6b8 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:01.328174 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838c85b-df0b-4d59-8323-ab9d9351c6b8 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:01.328445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838c85b-df0b-4d59-8323-ab9d9351c6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:01.349184 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 536/1058] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:39:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 08:39:01.353747 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-8838c85b-df0b-4d59-8323-ab9d9351c6b8 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:39:01.357487 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8838c85b-df0b-4d59-8323-ab9d9351c6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:39:01.357656 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d3245-14d5-4093-a389-b3244a87981f None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:01.407306 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:01.449793 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 523/1059] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:39:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 08:39:01.492960 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d3245-14d5-4093-a389-b3244a87981f None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:01.531366 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df071d-b9eb-453f-81eb-fbe15d6a9375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:01.532297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df071d-b9eb-453f-81eb-fbe15d6a9375 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:01.532713 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df071d-b9eb-453f-81eb-fbe15d6a9375 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:01.533123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df071d-b9eb-453f-81eb-fbe15d6a9375 None admin] PATH_INFO: `/v3/projects/bb116dbc16b74908bd04d1ae68630acf` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:01.534424 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df071d-b9eb-453f-81eb-fbe15d6a9375 None admin] RBAC: Authorizing `identity:get_project(project_id=bb116dbc16b74908bd04d1ae68630acf)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:01.536064 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df071d-b9eb-453f-81eb-fbe15d6a9375 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:01.538369 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 537/1060] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:39:01 2022] GET /identity/v3/projects/bb116dbc16b74908bd04d1ae68630acf => generated 311 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:39:01.544210 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838c85b-df0b-4d59-8323-ab9d9351c6b8 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:01.625735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:01.658423 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838c85b-df0b-4d59-8323-ab9d9351c6b8 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:01.702641 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35de2fc-1404-41e5-8bb7-ffb97fa7c23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:01.703294 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35de2fc-1404-41e5-8bb7-ffb97fa7c23a None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:01.703540 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35de2fc-1404-41e5-8bb7-ffb97fa7c23a None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:01.703830 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35de2fc-1404-41e5-8bb7-ffb97fa7c23a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:01.704554 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35de2fc-1404-41e5-8bb7-ffb97fa7c23a None admin] RBAC: Adding query filter params (role.id=18c3c8a821b647f7982a955880bd0a9a, scope.project.id=bb116dbc16b74908bd04d1ae68630acf, user.id=de5add03e9eb46e1ba04ce0121d21dd7) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:01.704855 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35de2fc-1404-41e5-8bb7-ffb97fa7c23a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:01.706285 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35de2fc-1404-41e5-8bb7-ffb97fa7c23a None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:01.718303 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 524/1061] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:39:01 2022] GET /identity/v3/role_assignments?user.id=de5add03e9eb46e1ba04ce0121d21dd7&scope.project.id=bb116dbc16b74908bd04d1ae68630acf&role.id=18c3c8a821b647f7982a955880bd0a9a => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:01.718555 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df071d-b9eb-453f-81eb-fbe15d6a9375 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:01.835630 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df071d-b9eb-453f-81eb-fbe15d6a9375 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:01.844066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:01.886212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8d8b3-ebdd-419c-a249-5bccc2c62214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:01.887006 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8d8b3-ebdd-419c-a249-5bccc2c62214 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:01.887204 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8d8b3-ebdd-419c-a249-5bccc2c62214 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:01.887564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8d8b3-ebdd-419c-a249-5bccc2c62214 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:01.901344 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8d8b3-ebdd-419c-a249-5bccc2c62214 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:39:01.906344 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-12a8d8b3-ebdd-419c-a249-5bccc2c62214 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 08:39:01.906344 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:39:01.906344 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:39:01.906344 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:39:01.906344 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:39:01.906344 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:39:01.906344 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:39:01.906344 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:39:01.906344 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:39:01.906344 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:39:01.906344 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:39:01.906344 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:39:01.906344 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:39:01.906344 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:39:01.906344 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:39:01.906344 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:39:01.907708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:39:01.907708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:39:01.907708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:39:01.907708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:39:01.907708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:39:01.907708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:39:01.907708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:39:01.907708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:39:01.907708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:39:01.907708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:39:01.907708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:39:01.907708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:39:01.907708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:39:01.907708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:39:01.907708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:39:01.907708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:39:01.909168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:39:01.909168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:39:01.909168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:39:01.909168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:39:01.909168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:39:01.909168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:39:01.909168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:39:01.909168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:39:01.909168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:39:01.909168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:39:01.909168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 08:39:01.909168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:39:01.909168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 538/1062] 10.222.0.71 () {56 vars in 1120 bytes} [Fri Aug 26 08:39:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:39:01.913860 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35de2fc-1404-41e5-8bb7-ffb97fa7c23a None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:01.996856 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc094d69-fedf-4eef-8f40-e995375b4d06 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:01.997137 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc094d69-fedf-4eef-8f40-e995375b4d06 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:01.997416 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc094d69-fedf-4eef-8f40-e995375b4d06 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:01.998826 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 539/1063] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:39:02.004321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075553ad-ec2f-433a-a2a8-9c164f7b8030 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:02.004606 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075553ad-ec2f-433a-a2a8-9c164f7b8030 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:02.004885 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075553ad-ec2f-433a-a2a8-9c164f7b8030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:02.024211 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-075553ad-ec2f-433a-a2a8-9c164f7b8030 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:39:02.025431 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-075553ad-ec2f-433a-a2a8-9c164f7b8030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:39:02.033760 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35de2fc-1404-41e5-8bb7-ffb97fa7c23a None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:02.071140 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407910e-2f24-4f60-841b-c477ecc4b0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:02.071898 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407910e-2f24-4f60-841b-c477ecc4b0bd None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:02.072177 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407910e-2f24-4f60-841b-c477ecc4b0bd None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:02.072444 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407910e-2f24-4f60-841b-c477ecc4b0bd None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:02.073129 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3407910e-2f24-4f60-841b-c477ecc4b0bd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:02.073429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3407910e-2f24-4f60-841b-c477ecc4b0bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:02.074814 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3407910e-2f24-4f60-841b-c477ecc4b0bd None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:02.077541 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:02.089089 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 525/1064] 10.222.0.71 () {56 vars in 1115 bytes} [Fri Aug 26 08:39:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:02.094554 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407910e-2f24-4f60-841b-c477ecc4b0bd None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:02.118159 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 540/1065] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:39:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 08:39:02.212107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407910e-2f24-4f60-841b-c477ecc4b0bd None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:02.251589 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56567a04-55b3-47cd-b85f-499622a7ac50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:02.252229 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56567a04-55b3-47cd-b85f-499622a7ac50 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:02.252496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56567a04-55b3-47cd-b85f-499622a7ac50 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:02.252765 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56567a04-55b3-47cd-b85f-499622a7ac50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:02.264203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56567a04-55b3-47cd-b85f-499622a7ac50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:02.265665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56567a04-55b3-47cd-b85f-499622a7ac50 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:02.281668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-56567a04-55b3-47cd-b85f-499622a7ac50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:39:02.281668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:39:02.281668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:39:02.281668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:39:02.281668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:39:02.281668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:39:02.281668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:39:02.281668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:39:02.281668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:39:02.281668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:39:02.281668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:39:02.281668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:39:02.281668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:39:02.281668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:39:02.281668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:39:02.281668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:39:02.282770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:39:02.282770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:39:02.282770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:39:02.282770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:39:02.282770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:39:02.282770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:39:02.282770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:39:02.282770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:39:02.282770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:39:02.282770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:39:02.282770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:39:02.282770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:39:02.282770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:39:02.282770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:39:02.282770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:39:02.282770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:39:02.282770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:39:02.283719 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:39:02.283719 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:39:02.283719 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:39:02.287257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 526/1066] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:39:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:39:02.288999 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075553ad-ec2f-433a-a2a8-9c164f7b8030 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:02.387204 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56567a04-55b3-47cd-b85f-499622a7ac50 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:02.409848 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:02.418433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075553ad-ec2f-433a-a2a8-9c164f7b8030 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:02.446788 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:02.465610 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a9d13-0797-41d3-9428-27dc88b8ef05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:02.466258 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a9d13-0797-41d3-9428-27dc88b8ef05 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:02.466536 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a9d13-0797-41d3-9428-27dc88b8ef05 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:02.466809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a9d13-0797-41d3-9428-27dc88b8ef05 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:02.467514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a9d13-0797-41d3-9428-27dc88b8ef05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:02.467821 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a9d13-0797-41d3-9428-27dc88b8ef05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:02.469236 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a9d13-0797-41d3-9428-27dc88b8ef05 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:02.483638 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 541/1067] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:02.489650 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a9d13-0797-41d3-9428-27dc88b8ef05 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:02.507225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56567a04-55b3-47cd-b85f-499622a7ac50 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:02.550004 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520125a-f0ee-4763-b2c0-439965274284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:02.550647 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520125a-f0ee-4763-b2c0-439965274284 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:02.550981 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520125a-f0ee-4763-b2c0-439965274284 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:02.551368 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520125a-f0ee-4763-b2c0-439965274284 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:02.631298 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a9d13-0797-41d3-9428-27dc88b8ef05 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:02.639660 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520125a-f0ee-4763-b2c0-439965274284 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:02.641151 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520125a-f0ee-4763-b2c0-439965274284 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:02.673794 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73239d08-4684-4394-aef8-e79fcbe8a449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:02.674449 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73239d08-4684-4394-aef8-e79fcbe8a449 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:02.674804 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73239d08-4684-4394-aef8-e79fcbe8a449 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:02.675040 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73239d08-4684-4394-aef8-e79fcbe8a449 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:02.683812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73239d08-4684-4394-aef8-e79fcbe8a449 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:02.685310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73239d08-4684-4394-aef8-e79fcbe8a449 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:02.697949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-73239d08-4684-4394-aef8-e79fcbe8a449 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 08:39:02.697949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:39:02.697949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:39:02.697949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:39:02.697949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:39:02.697949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:39:02.697949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:39:02.697949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:39:02.697949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:39:02.697949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:39:02.697949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:39:02.697949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:39:02.697949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:39:02.697949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:39:02.697949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:39:02.697949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:39:02.699255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:39:02.699255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:39:02.699255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:39:02.699255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:39:02.699255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:39:02.699255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:39:02.699255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:39:02.699255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:39:02.699255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:39:02.699255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:39:02.699255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:39:02.699255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:39:02.699255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:39:02.699255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:39:02.699255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:39:02.699255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:39:02.699255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:39:02.699255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:39:02.700227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:39:02.700227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:39:02.700227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:39:02.700227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:39:02.700227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:39:02.700227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:39:02.700227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:39:02.700227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:39:02.700227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 08:39:02.700227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:39:02.700227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 542/1068] 10.222.0.71 () {56 vars in 1093 bytes} [Fri Aug 26 08:39:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:39:02.701779 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:02.707397 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73239d08-4684-4394-aef8-e79fcbe8a449 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:02.728479 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:02.775314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 527/1069] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:39:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 08:39:02.782708 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520125a-f0ee-4763-b2c0-439965274284 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:02.853635 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73239d08-4684-4394-aef8-e79fcbe8a449 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:02.896374 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0939f88-e604-41ef-94e4-6ad59667c727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:02.896994 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0939f88-e604-41ef-94e4-6ad59667c727 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:02.897286 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0939f88-e604-41ef-94e4-6ad59667c727 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:02.897613 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0939f88-e604-41ef-94e4-6ad59667c727 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:02.899272 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0939f88-e604-41ef-94e4-6ad59667c727 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:02.899733 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0939f88-e604-41ef-94e4-6ad59667c727 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:02.902321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0939f88-e604-41ef-94e4-6ad59667c727 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:02.913663 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520125a-f0ee-4763-b2c0-439965274284 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:02.923894 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 543/1070] 10.222.0.71 () {56 vars in 1133 bytes} [Fri Aug 26 08:39:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:02.930701 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0939f88-e604-41ef-94e4-6ad59667c727 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:02.964029 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e5990c-0048-43a9-8f64-1a95bf8fcdfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:02.964698 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5990c-0048-43a9-8f64-1a95bf8fcdfd None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:02.965089 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5990c-0048-43a9-8f64-1a95bf8fcdfd None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:02.965390 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5990c-0048-43a9-8f64-1a95bf8fcdfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:02.976739 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e5990c-0048-43a9-8f64-1a95bf8fcdfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:02.978261 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e5990c-0048-43a9-8f64-1a95bf8fcdfd None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:02.995912 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-65e5990c-0048-43a9-8f64-1a95bf8fcdfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:39:02.995912 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:39:02.995912 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:39:02.995912 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:39:02.995912 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:39:02.995912 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:39:02.995912 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:39:02.995912 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:39:02.995912 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:39:02.995912 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:39:02.995912 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:39:02.995912 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:39:02.995912 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:39:02.995912 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:39:02.995912 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:39:02.995912 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:39:02.997257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:39:02.997257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:39:02.997257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:39:02.997257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:39:02.997257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:39:02.997257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:39:02.997257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:39:02.997257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:39:02.997257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:39:02.997257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:39:02.997257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:39:02.997257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:39:02.997257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:39:02.997257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:39:02.997257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:39:02.997257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:39:02.997257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:39:02.998158 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:39:02.998158 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:39:02.998158 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:39:03.003061 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 528/1071] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:39:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:39:03.004917 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e5990c-0048-43a9-8f64-1a95bf8fcdfd None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:03.009408 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:03.022289 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:03.048283 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0939f88-e604-41ef-94e4-6ad59667c727 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:03.088168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a48a12-3301-486e-8614-c359633fd3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:03.088979 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a48a12-3301-486e-8614-c359633fd3f7 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:03.089264 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a48a12-3301-486e-8614-c359633fd3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:03.089541 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a48a12-3301-486e-8614-c359633fd3f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:03.102279 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a48a12-3301-486e-8614-c359633fd3f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:03.103833 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a48a12-3301-486e-8614-c359633fd3f7 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:03.123814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-00a48a12-3301-486e-8614-c359633fd3f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:39:03.123814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:39:03.123814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:39:03.123814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:39:03.123814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:39:03.123814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:39:03.123814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:39:03.123814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:39:03.123814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:39:03.123814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:39:03.123814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:39:03.123814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:39:03.123814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:39:03.123814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:39:03.123814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:39:03.123814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:39:03.124847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:39:03.124847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:39:03.124847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:39:03.124847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:39:03.124847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:39:03.124847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:39:03.124847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:39:03.124847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:39:03.124847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:39:03.124847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:39:03.124847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:39:03.124847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:39:03.124847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:39:03.124847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:39:03.124847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:39:03.124847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:39:03.124847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:39:03.125758 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:39:03.125758 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:39:03.125758 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:39:03.125758 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 544/1072] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:39:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:39:03.131112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a48a12-3301-486e-8614-c359633fd3f7 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:03.143069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e5990c-0048-43a9-8f64-1a95bf8fcdfd None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:03.187116 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74b4b6-d579-4ac1-98c2-320f35385191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:03.188104 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74b4b6-d579-4ac1-98c2-320f35385191 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:03.188376 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74b4b6-d579-4ac1-98c2-320f35385191 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:03.188644 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74b4b6-d579-4ac1-98c2-320f35385191 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:03.189323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f74b4b6-d579-4ac1-98c2-320f35385191 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:03.189731 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f74b4b6-d579-4ac1-98c2-320f35385191 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:03.191224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f74b4b6-d579-4ac1-98c2-320f35385191 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:03.208800 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 529/1073] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:39:03.256479 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a48a12-3301-486e-8614-c359633fd3f7 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:03.294084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:03.309680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15cb59-b7f2-4be2-ba07-ba56e695cc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:03.310694 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15cb59-b7f2-4be2-ba07-ba56e695cc75 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:03.311150 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15cb59-b7f2-4be2-ba07-ba56e695cc75 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:03.311582 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15cb59-b7f2-4be2-ba07-ba56e695cc75 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:03.312757 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15cb59-b7f2-4be2-ba07-ba56e695cc75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:03.313316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15cb59-b7f2-4be2-ba07-ba56e695cc75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:03.315818 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15cb59-b7f2-4be2-ba07-ba56e695cc75 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:03.337575 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 545/1074] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:39:03.339066 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74b4b6-d579-4ac1-98c2-320f35385191 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:03.430721 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:03.476792 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74b4b6-d579-4ac1-98c2-320f35385191 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:03.519401 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a872b-85bd-4929-a10b-527f35c53a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:03.520273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a872b-85bd-4929-a10b-527f35c53a83 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:03.520459 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a872b-85bd-4929-a10b-527f35c53a83 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:03.520627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a872b-85bd-4929-a10b-527f35c53a83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:03.528211 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7a872b-85bd-4929-a10b-527f35c53a83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:03.529690 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7a872b-85bd-4929-a10b-527f35c53a83 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:03.542314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-fa7a872b-85bd-4929-a10b-527f35c53a83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:39:03.542314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:39:03.542314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:39:03.542314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:39:03.542314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:39:03.542314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:39:03.542314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:39:03.542314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:39:03.542314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:39:03.542314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:39:03.542314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:39:03.542314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:39:03.542314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:39:03.542314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:39:03.542314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:39:03.543533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:39:03.543533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:39:03.543533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:39:03.543533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:39:03.543533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:39:03.543533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:39:03.543533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:39:03.543533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:39:03.543533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:39:03.543533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:39:03.543533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:39:03.543533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:39:03.543533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:39:03.543533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:39:03.543533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:39:03.543533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:39:03.543533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:39:03.544532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:39:03.544532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:39:03.544532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:39:03.544532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:39:03.544532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:39:03.544532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:39:03.544532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:39:03.544532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:39:03.544532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 530/1075] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:39:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:39:03.550075 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15cb59-b7f2-4be2-ba07-ba56e695cc75 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:03.681178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15cb59-b7f2-4be2-ba07-ba56e695cc75 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:03.688820 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d71361-3b1b-4206-a04f-7c0f278ad02a None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:03.689082 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d71361-3b1b-4206-a04f-7c0f278ad02a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:03.689368 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d71361-3b1b-4206-a04f-7c0f278ad02a None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:03.690862 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 531/1076] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:39:03.696273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd6ddc-40c1-46a7-8073-acc9234381c9 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:03.696680 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd6ddc-40c1-46a7-8073-acc9234381c9 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:03.696822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd6ddc-40c1-46a7-8073-acc9234381c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:03.715577 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-badd6ddc-40c1-46a7-8073-acc9234381c9 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:39:03.716778 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-badd6ddc-40c1-46a7-8073-acc9234381c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:39:03.718810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cf9d9-df6d-4051-ae94-75563ff5418d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:03.719477 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cf9d9-df6d-4051-ae94-75563ff5418d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:03.719756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cf9d9-df6d-4051-ae94-75563ff5418d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:03.720031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cf9d9-df6d-4051-ae94-75563ff5418d None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:03.720745 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8cf9d9-df6d-4051-ae94-75563ff5418d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:03.721049 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8cf9d9-df6d-4051-ae94-75563ff5418d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:03.722500 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8cf9d9-df6d-4051-ae94-75563ff5418d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:03.734995 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 546/1077] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:39:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:03.741914 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cf9d9-df6d-4051-ae94-75563ff5418d None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:03.791161 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:03.791651 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 532/1078] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:39:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 08:39:03.848694 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cf9d9-df6d-4051-ae94-75563ff5418d None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:03.891161 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f2d4f-46f6-4067-819e-568487b99f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:03.891823 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f2d4f-46f6-4067-819e-568487b99f27 None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:03.892101 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f2d4f-46f6-4067-819e-568487b99f27 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:03.892375 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f2d4f-46f6-4067-819e-568487b99f27 None admin] PATH_INFO: `/v3/projects/543d44293d894a7fbb83401f7a26dd9b/users/f16d1768715a41648af3fe6afadd7477/roles/f795ff29508a4109be2917083c5ead38` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:03.894583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f2d4f-46f6-4067-819e-568487b99f27 None admin] RBAC: Authorizing `identity:create_grant(project_id=543d44293d894a7fbb83401f7a26dd9b, user_id=f16d1768715a41648af3fe6afadd7477, role_id=f795ff29508a4109be2917083c5ead38)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:03.896117 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f2d4f-46f6-4067-819e-568487b99f27 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:03.910557 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 547/1079] 10.222.0.71 () {58 vars in 1439 bytes} [Fri Aug 26 08:39:03 2022] PUT /identity/v3/projects/543d44293d894a7fbb83401f7a26dd9b/users/f16d1768715a41648af3fe6afadd7477/roles/f795ff29508a4109be2917083c5ead38 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:39:04.058299 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd6ddc-40c1-46a7-8073-acc9234381c9 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:04.130420 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:04.183394 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd6ddc-40c1-46a7-8073-acc9234381c9 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:04.221898 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae68e2-1b39-439e-b123-7bdafdfd0b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:04.222545 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae68e2-1b39-439e-b123-7bdafdfd0b92 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:04.222819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae68e2-1b39-439e-b123-7bdafdfd0b92 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:04.223115 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae68e2-1b39-439e-b123-7bdafdfd0b92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:04.235488 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acae68e2-1b39-439e-b123-7bdafdfd0b92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:39:04.240327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-acae68e2-1b39-439e-b123-7bdafdfd0b92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:39:04.240327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:39:04.240327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:39:04.240327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:39:04.240327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:39:04.240327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:39:04.240327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:39:04.240327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:39:04.240327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:39:04.240327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:39:04.240327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:39:04.240327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:39:04.240327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:39:04.240327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:39:04.240327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:39:04.240327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:39:04.241743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:39:04.241743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:39:04.241743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:39:04.241743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:39:04.241743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:39:04.241743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:39:04.241743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:39:04.241743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:39:04.241743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:39:04.241743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:39:04.241743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:39:04.241743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:39:04.241743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:39:04.241743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:39:04.241743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:39:04.241743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:39:04.242761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:39:04.242761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:39:04.242761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:39:04.242761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:39:04.242761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:39:04.242761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:39:04.242761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:39:04.242761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:39:04.242761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:39:04.242761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:39:04.242761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:39:04.242761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:39:04.247055 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 533/1080] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:39:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:39:04.248554 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f2d4f-46f6-4067-819e-568487b99f27 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:04.360405 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f2d4f-46f6-4067-819e-568487b99f27 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:04.368234 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:04.403956 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82b59d-eacb-445c-86c3-fad84f04ca9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:04.404576 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82b59d-eacb-445c-86c3-fad84f04ca9a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:04.404845 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82b59d-eacb-445c-86c3-fad84f04ca9a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:04.405106 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82b59d-eacb-445c-86c3-fad84f04ca9a None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:04.405780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa82b59d-eacb-445c-86c3-fad84f04ca9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:04.406076 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa82b59d-eacb-445c-86c3-fad84f04ca9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:04.407488 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa82b59d-eacb-445c-86c3-fad84f04ca9a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:04.416448 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 548/1081] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:39:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:04.422517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae68e2-1b39-439e-b123-7bdafdfd0b92 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:04.541492 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae68e2-1b39-439e-b123-7bdafdfd0b92 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:04.557739 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:04.582846 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598c3f2-d1e6-46fa-88fe-53287f5c4c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:04.583512 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598c3f2-d1e6-46fa-88fe-53287f5c4c03 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:04.583783 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598c3f2-d1e6-46fa-88fe-53287f5c4c03 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:04.584036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598c3f2-d1e6-46fa-88fe-53287f5c4c03 None admin] PATH_INFO: `/v3/users/de5add03e9eb46e1ba04ce0121d21dd7` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:04.585378 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2598c3f2-d1e6-46fa-88fe-53287f5c4c03 None admin] RBAC: Authorizing `identity:get_user(user_id=de5add03e9eb46e1ba04ce0121d21dd7)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:04.586831 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2598c3f2-d1e6-46fa-88fe-53287f5c4c03 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:04.588495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 534/1082] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:39:04 2022] GET /identity/v3/users/de5add03e9eb46e1ba04ce0121d21dd7 => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:04.594448 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82b59d-eacb-445c-86c3-fad84f04ca9a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:04.717297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82b59d-eacb-445c-86c3-fad84f04ca9a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:04.752248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341d88c-f82b-4a0e-a7f3-37e14b930e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:04.752925 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341d88c-f82b-4a0e-a7f3-37e14b930e7f None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:04.753147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341d88c-f82b-4a0e-a7f3-37e14b930e7f None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:04.753414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341d88c-f82b-4a0e-a7f3-37e14b930e7f None admin] PATH_INFO: `/v3/projects/bb116dbc16b74908bd04d1ae68630acf` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:04.754536 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6341d88c-f82b-4a0e-a7f3-37e14b930e7f None admin] RBAC: Authorizing `identity:get_project(project_id=bb116dbc16b74908bd04d1ae68630acf)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:04.756005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6341d88c-f82b-4a0e-a7f3-37e14b930e7f None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:04.757845 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 549/1083] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:39:04 2022] GET /identity/v3/projects/bb116dbc16b74908bd04d1ae68630acf => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:04.764112 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598c3f2-d1e6-46fa-88fe-53287f5c4c03 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:04.880338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598c3f2-d1e6-46fa-88fe-53287f5c4c03 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:04.919543 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d9bad-305f-43a1-bbf7-5aa3059cfd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:04.920227 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d9bad-305f-43a1-bbf7-5aa3059cfd5c None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:04.920499 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d9bad-305f-43a1-bbf7-5aa3059cfd5c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:04.920767 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d9bad-305f-43a1-bbf7-5aa3059cfd5c None admin] PATH_INFO: `/v3/projects/bb116dbc16b74908bd04d1ae68630acf/users/de5add03e9eb46e1ba04ce0121d21dd7/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:04.922952 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d9bad-305f-43a1-bbf7-5aa3059cfd5c None admin] RBAC: Authorizing `identity:create_grant(project_id=bb116dbc16b74908bd04d1ae68630acf, user_id=de5add03e9eb46e1ba04ce0121d21dd7, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:04.924456 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d9bad-305f-43a1-bbf7-5aa3059cfd5c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:04.949077 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 535/1084] 10.222.0.71 () {58 vars in 1439 bytes} [Fri Aug 26 08:39:04 2022] PUT /identity/v3/projects/bb116dbc16b74908bd04d1ae68630acf/users/de5add03e9eb46e1ba04ce0121d21dd7/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:39:05.830417 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc594b0-9da9-464b-87e5-1a0eb7acd62a None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:05.831038 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc594b0-9da9-464b-87e5-1a0eb7acd62a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:05.831038 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc594b0-9da9-464b-87e5-1a0eb7acd62a None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:05.832343 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 550/1085] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:39:05.837902 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519b5c7-e0b1-444b-90d1-3c51b92b294a None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:05.838461 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519b5c7-e0b1-444b-90d1-3c51b92b294a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:05.838461 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519b5c7-e0b1-444b-90d1-3c51b92b294a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:05.863104 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-2519b5c7-e0b1-444b-90d1-3c51b92b294a None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:39:05.864865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-2519b5c7-e0b1-444b-90d1-3c51b92b294a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:39:05.914973 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:05.965540 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 536/1086] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:39:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:39:06.225824 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc594b0-9da9-464b-87e5-1a0eb7acd62a None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:06.271059 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:06.340480 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc594b0-9da9-464b-87e5-1a0eb7acd62a None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:06.384815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b25454-cc9e-47d7-a9a1-e2bd8c3e5e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:06.385487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b25454-cc9e-47d7-a9a1-e2bd8c3e5e50 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:06.385768 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b25454-cc9e-47d7-a9a1-e2bd8c3e5e50 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:06.386041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b25454-cc9e-47d7-a9a1-e2bd8c3e5e50 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:06.398562 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b25454-cc9e-47d7-a9a1-e2bd8c3e5e50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:39:06.403438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-27b25454-cc9e-47d7-a9a1-e2bd8c3e5e50 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 08:39:06.403438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:39:06.403438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:39:06.403438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:39:06.403438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:39:06.403438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:39:06.403438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:39:06.403438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:39:06.403438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:39:06.403438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:39:06.403438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:39:06.403438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:39:06.403438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:39:06.403438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:39:06.403438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:39:06.403438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:39:06.404499 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:39:06.404499 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:39:06.404499 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:39:06.404499 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:39:06.404499 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:39:06.404499 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:39:06.404499 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:39:06.404499 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:39:06.404499 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:39:06.404499 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:39:06.404499 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:39:06.404499 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:39:06.404499 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:39:06.404499 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:39:06.404499 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:39:06.404499 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:39:06.405383 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:39:06.405383 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:39:06.405383 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:39:06.405383 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:39:06.405383 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:39:06.405383 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:39:06.405383 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:39:06.405383 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:39:06.405383 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:39:06.405383 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:39:06.405383 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 08:39:06.405383 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:39:06.405383 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 551/1087] 10.222.0.71 () {56 vars in 1120 bytes} [Fri Aug 26 08:39:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:39:06.411029 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519b5c7-e0b1-444b-90d1-3c51b92b294a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:06.519142 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519b5c7-e0b1-444b-90d1-3c51b92b294a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:06.555222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3dc5d-a211-4cb4-a354-63dda17c77ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:06.555893 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3dc5d-a211-4cb4-a354-63dda17c77ac None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:06.556171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3dc5d-a211-4cb4-a354-63dda17c77ac None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:06.556442 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3dc5d-a211-4cb4-a354-63dda17c77ac None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:06.557138 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa3dc5d-a211-4cb4-a354-63dda17c77ac None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:06.557446 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa3dc5d-a211-4cb4-a354-63dda17c77ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:06.558850 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa3dc5d-a211-4cb4-a354-63dda17c77ac None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:06.567742 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 537/1088] 10.222.0.71 () {56 vars in 1115 bytes} [Fri Aug 26 08:39:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:06.573870 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b25454-cc9e-47d7-a9a1-e2bd8c3e5e50 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:06.675724 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:06.683708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b25454-cc9e-47d7-a9a1-e2bd8c3e5e50 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:06.721163 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a375d93-4b51-44b4-868d-b8a6029154f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:06.721804 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a375d93-4b51-44b4-868d-b8a6029154f4 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:06.722082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a375d93-4b51-44b4-868d-b8a6029154f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:06.722355 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a375d93-4b51-44b4-868d-b8a6029154f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:06.734615 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a375d93-4b51-44b4-868d-b8a6029154f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:06.736279 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a375d93-4b51-44b4-868d-b8a6029154f4 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:06.753257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-5a375d93-4b51-44b4-868d-b8a6029154f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:39:06.753257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:39:06.753257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:39:06.753257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:39:06.753257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:39:06.753257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:39:06.753257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:39:06.753257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:39:06.753257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:39:06.753257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:39:06.753257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:39:06.753257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:39:06.753257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:39:06.753257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:39:06.753257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:39:06.753257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:39:06.754419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:39:06.754419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:39:06.754419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:39:06.754419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:39:06.754419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:39:06.754419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:39:06.754419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:39:06.754419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:39:06.754419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:39:06.754419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:39:06.754419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:39:06.754419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:39:06.754419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:39:06.754419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:39:06.754419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:39:06.754419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:39:06.754419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:39:06.755612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:39:06.755612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:39:06.755612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:39:06.755612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 552/1089] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:39:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:39:06.762024 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3dc5d-a211-4cb4-a354-63dda17c77ac None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:06.798998 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:06.873523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e7ea2-96ab-4746-9553-1f6d28152741 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:06.874142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e7ea2-96ab-4746-9553-1f6d28152741 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:06.874142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e7ea2-96ab-4746-9553-1f6d28152741 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:06.875603 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 553/1090] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:39:06.880871 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31aada-ad9a-48ca-958d-e1f1514c5f55 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:06.881967 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31aada-ad9a-48ca-958d-e1f1514c5f55 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:06.881967 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31aada-ad9a-48ca-958d-e1f1514c5f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:06.882797 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3dc5d-a211-4cb4-a354-63dda17c77ac None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:06.903508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-3a31aada-ad9a-48ca-958d-e1f1514c5f55 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:39:06.905634 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3a31aada-ad9a-48ca-958d-e1f1514c5f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:39:06.921572 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92382b7-c008-43f9-b438-dc4ba169df76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:06.922216 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92382b7-c008-43f9-b438-dc4ba169df76 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:06.922493 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92382b7-c008-43f9-b438-dc4ba169df76 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:06.922758 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92382b7-c008-43f9-b438-dc4ba169df76 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:06.923455 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92382b7-c008-43f9-b438-dc4ba169df76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:06.923760 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92382b7-c008-43f9-b438-dc4ba169df76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:06.925169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92382b7-c008-43f9-b438-dc4ba169df76 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:06.937509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 538/1091] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:39:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:06.943547 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92382b7-c008-43f9-b438-dc4ba169df76 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:06.980545 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 554/1092] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:39:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:39:07.043736 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:07.052950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92382b7-c008-43f9-b438-dc4ba169df76 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:07.091599 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc568a7-d50f-4020-83ae-10f62009c648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:07.092241 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc568a7-d50f-4020-83ae-10f62009c648 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:07.092518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc568a7-d50f-4020-83ae-10f62009c648 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:07.092788 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc568a7-d50f-4020-83ae-10f62009c648 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:07.100373 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc568a7-d50f-4020-83ae-10f62009c648 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:07.101978 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc568a7-d50f-4020-83ae-10f62009c648 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:07.114611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-8bc568a7-d50f-4020-83ae-10f62009c648 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 08:39:07.114611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:39:07.114611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:39:07.114611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:39:07.114611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:39:07.114611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:39:07.114611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:39:07.114611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:39:07.114611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:39:07.114611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:39:07.114611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:39:07.114611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:39:07.114611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:39:07.114611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:39:07.114611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:39:07.114611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:39:07.115665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:39:07.115665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:39:07.115665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:39:07.115665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:39:07.115665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:39:07.115665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:39:07.115665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:39:07.115665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:39:07.115665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:39:07.115665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:39:07.115665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:39:07.115665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:39:07.115665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:39:07.115665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:39:07.115665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:39:07.115665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:39:07.115665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:39:07.115665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:39:07.116648 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:39:07.116648 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:39:07.116648 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:39:07.116648 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:39:07.116648 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:39:07.116648 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:39:07.116648 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:39:07.116648 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:39:07.116648 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 08:39:07.116648 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:39:07.116648 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 539/1093] 10.222.0.71 () {56 vars in 1093 bytes} [Fri Aug 26 08:39:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:39:07.122070 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31aada-ad9a-48ca-958d-e1f1514c5f55 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:07.203751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:07.226254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31aada-ad9a-48ca-958d-e1f1514c5f55 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:07.240857 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc568a7-d50f-4020-83ae-10f62009c648 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:07.262064 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b1d821-a1b1-4650-bfe5-cf6b51021c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:07.262693 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1d821-a1b1-4650-bfe5-cf6b51021c69 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:07.262996 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1d821-a1b1-4650-bfe5-cf6b51021c69 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:07.263268 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1d821-a1b1-4650-bfe5-cf6b51021c69 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:07.264017 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b1d821-a1b1-4650-bfe5-cf6b51021c69 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:07.264320 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b1d821-a1b1-4650-bfe5-cf6b51021c69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:07.265744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b1d821-a1b1-4650-bfe5-cf6b51021c69 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:07.284007 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 555/1094] 10.222.0.71 () {56 vars in 1133 bytes} [Fri Aug 26 08:39:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:07.290155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b1d821-a1b1-4650-bfe5-cf6b51021c69 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:07.348507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc568a7-d50f-4020-83ae-10f62009c648 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:07.386077 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab25dc-0997-42e5-9434-4ceac5389de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:07.386726 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab25dc-0997-42e5-9434-4ceac5389de8 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:07.387047 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab25dc-0997-42e5-9434-4ceac5389de8 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:07.387322 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab25dc-0997-42e5-9434-4ceac5389de8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:07.396915 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b1d821-a1b1-4650-bfe5-cf6b51021c69 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:07.398480 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ab25dc-0997-42e5-9434-4ceac5389de8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:39:07.403222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-46ab25dc-0997-42e5-9434-4ceac5389de8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:39:07.403222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:39:07.403222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:39:07.403222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:39:07.403222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:39:07.403222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:39:07.403222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:39:07.403222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:39:07.403222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:39:07.403222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:39:07.403222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:39:07.403222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:39:07.403222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:39:07.403222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:39:07.403222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:39:07.403222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:39:07.404338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:39:07.404338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:39:07.404338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:39:07.404338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:39:07.404338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:39:07.404338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:39:07.404338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:39:07.404338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:39:07.404338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:39:07.404338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:39:07.404338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:39:07.404338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:39:07.404338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:39:07.404338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:39:07.404338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:39:07.404338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:39:07.405265 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:39:07.405265 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:39:07.405265 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:39:07.405265 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:39:07.405265 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:39:07.405265 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:39:07.405265 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:39:07.405265 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:39:07.405265 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:39:07.405265 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:39:07.405265 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:39:07.405265 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:39:07.405265 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 540/1095] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:39:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:39:07.410649 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab25dc-0997-42e5-9434-4ceac5389de8 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:07.427084 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:07.439026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a18692-c1b1-4abd-9a69-888408ef9c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:07.439642 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a18692-c1b1-4abd-9a69-888408ef9c7b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:07.439924 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a18692-c1b1-4abd-9a69-888408ef9c7b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:07.440194 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a18692-c1b1-4abd-9a69-888408ef9c7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:07.444784 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:07.521084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a18692-c1b1-4abd-9a69-888408ef9c7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:07.522552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a18692-c1b1-4abd-9a69-888408ef9c7b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:07.529324 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab25dc-0997-42e5-9434-4ceac5389de8 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:07.564005 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fdf3e-bccd-441d-a501-73b527b9e085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:07.564623 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fdf3e-bccd-441d-a501-73b527b9e085 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:07.564907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fdf3e-bccd-441d-a501-73b527b9e085 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:07.565175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fdf3e-bccd-441d-a501-73b527b9e085 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:07.565850 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446fdf3e-bccd-441d-a501-73b527b9e085 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:07.566152 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446fdf3e-bccd-441d-a501-73b527b9e085 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:07.567584 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446fdf3e-bccd-441d-a501-73b527b9e085 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:07.576544 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 541/1096] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:39:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:39:07.583092 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fdf3e-bccd-441d-a501-73b527b9e085 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:07.632299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 556/1097] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:39:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 08:39:07.638251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a18692-c1b1-4abd-9a69-888408ef9c7b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:07.663757 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:07.674232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:07.690754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fdf3e-bccd-441d-a501-73b527b9e085 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:07.730481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f8b38-f528-4fc1-8ff4-0ee781f990a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:07.731147 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f8b38-f528-4fc1-8ff4-0ee781f990a6 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:07.731421 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f8b38-f528-4fc1-8ff4-0ee781f990a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:07.731689 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f8b38-f528-4fc1-8ff4-0ee781f990a6 None admin] PATH_INFO: `/v3/users/de5add03e9eb46e1ba04ce0121d21dd7` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:07.732848 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859f8b38-f528-4fc1-8ff4-0ee781f990a6 None admin] RBAC: Authorizing `identity:get_user(user_id=de5add03e9eb46e1ba04ce0121d21dd7)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:07.734347 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859f8b38-f528-4fc1-8ff4-0ee781f990a6 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:07.735987 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 542/1098] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:39:07 2022] GET /identity/v3/users/de5add03e9eb46e1ba04ce0121d21dd7 => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:39:07.742185 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f8b38-f528-4fc1-8ff4-0ee781f990a6 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:07.747271 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a18692-c1b1-4abd-9a69-888408ef9c7b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:07.784705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16359f5d-ca49-4822-8254-1f820f6d2b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:07.785375 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16359f5d-ca49-4822-8254-1f820f6d2b0d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:07.785653 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16359f5d-ca49-4822-8254-1f820f6d2b0d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:07.785924 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16359f5d-ca49-4822-8254-1f820f6d2b0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:07.798933 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16359f5d-ca49-4822-8254-1f820f6d2b0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:07.800424 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16359f5d-ca49-4822-8254-1f820f6d2b0d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:07.815844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-16359f5d-ca49-4822-8254-1f820f6d2b0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:39:07.815844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:39:07.815844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:39:07.815844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:39:07.815844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:39:07.815844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:39:07.815844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:39:07.815844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:39:07.815844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:39:07.815844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:39:07.815844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:39:07.815844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:39:07.815844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:39:07.815844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:39:07.815844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:39:07.815844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:39:07.817066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:39:07.817066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:39:07.817066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:39:07.817066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:39:07.817066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:39:07.817066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:39:07.817066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:39:07.817066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:39:07.817066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:39:07.817066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:39:07.817066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:39:07.817066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:39:07.817066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:39:07.817066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:39:07.817066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:39:07.817066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:39:07.817066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:39:07.818371 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:39:07.818371 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:39:07.818371 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:39:07.818371 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 557/1099] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:39:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:39:07.823025 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16359f5d-ca49-4822-8254-1f820f6d2b0d None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:07.858045 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f8b38-f528-4fc1-8ff4-0ee781f990a6 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:07.894691 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213eaf2-e634-4c5f-8bb4-4bfe83454418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:07.895393 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213eaf2-e634-4c5f-8bb4-4bfe83454418 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:07.895515 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213eaf2-e634-4c5f-8bb4-4bfe83454418 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:07.895792 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213eaf2-e634-4c5f-8bb4-4bfe83454418 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:07.908472 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:07.923581 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:07.937895 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16359f5d-ca49-4822-8254-1f820f6d2b0d None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:07.970540 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213eaf2-e634-4c5f-8bb4-4bfe83454418 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:07.972081 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213eaf2-e634-4c5f-8bb4-4bfe83454418 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:07.977126 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630a116-e696-4e3c-89d1-a96da1ba9819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:07.977757 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630a116-e696-4e3c-89d1-a96da1ba9819 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:07.978044 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630a116-e696-4e3c-89d1-a96da1ba9819 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:07.978316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630a116-e696-4e3c-89d1-a96da1ba9819 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:07.979423 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c630a116-e696-4e3c-89d1-a96da1ba9819 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:07.979423 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c630a116-e696-4e3c-89d1-a96da1ba9819 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:07.980715 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c630a116-e696-4e3c-89d1-a96da1ba9819 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:07.993367 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 558/1100] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:39:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:07.999560 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630a116-e696-4e3c-89d1-a96da1ba9819 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:08.078111 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 543/1101] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:39:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 08:39:08.084203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213eaf2-e634-4c5f-8bb4-4bfe83454418 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:08.106913 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630a116-e696-4e3c-89d1-a96da1ba9819 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:08.129144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:08.146056 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa780b9c-6a2a-4627-9071-aed373e27140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:08.146646 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa780b9c-6a2a-4627-9071-aed373e27140 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:08.146955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa780b9c-6a2a-4627-9071-aed373e27140 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:08.147228 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa780b9c-6a2a-4627-9071-aed373e27140 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:08.154325 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa780b9c-6a2a-4627-9071-aed373e27140 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:08.155847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa780b9c-6a2a-4627-9071-aed373e27140 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:08.167366 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-aa780b9c-6a2a-4627-9071-aed373e27140 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:39:08.167366 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:39:08.167366 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:39:08.167366 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:39:08.167366 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:39:08.167366 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:39:08.167366 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:39:08.167366 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:39:08.167366 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:39:08.167366 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:39:08.167366 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:39:08.167366 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:39:08.167366 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:39:08.167366 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:39:08.167366 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:39:08.168612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:39:08.168612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:39:08.168612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:39:08.168612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:39:08.168612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:39:08.168612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:39:08.168612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:39:08.168612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:39:08.168612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:39:08.168612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:39:08.168612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:39:08.168612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:39:08.168612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:39:08.168612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:39:08.168612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:39:08.168612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:39:08.168612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:39:08.169549 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:39:08.169549 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:39:08.169549 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:39:08.169549 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:39:08.169549 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:39:08.169549 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:39:08.169549 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:39:08.169549 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:39:08.169549 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 559/1102] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:39:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:39:08.174652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa780b9c-6a2a-4627-9071-aed373e27140 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:08.186178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:08.192865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213eaf2-e634-4c5f-8bb4-4bfe83454418 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:08.228620 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90193e5e-ac16-4161-bfa7-7996f8501057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:08.229259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90193e5e-ac16-4161-bfa7-7996f8501057 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:08.229555 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90193e5e-ac16-4161-bfa7-7996f8501057 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:08.229829 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90193e5e-ac16-4161-bfa7-7996f8501057 None admin] PATH_INFO: `/v3/projects/bb116dbc16b74908bd04d1ae68630acf` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:08.231019 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90193e5e-ac16-4161-bfa7-7996f8501057 None admin] RBAC: Authorizing `identity:get_project(project_id=bb116dbc16b74908bd04d1ae68630acf)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:08.232487 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90193e5e-ac16-4161-bfa7-7996f8501057 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:08.234195 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 544/1103] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:39:08 2022] GET /identity/v3/projects/bb116dbc16b74908bd04d1ae68630acf => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:39:08.240878 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90193e5e-ac16-4161-bfa7-7996f8501057 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:08.284053 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa780b9c-6a2a-4627-9071-aed373e27140 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:08.321655 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2843c79-676e-434b-a5a9-897ae6ba6d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:08.322751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2843c79-676e-434b-a5a9-897ae6ba6d33 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:08.322751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2843c79-676e-434b-a5a9-897ae6ba6d33 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:08.323049 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2843c79-676e-434b-a5a9-897ae6ba6d33 None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:08.323583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2843c79-676e-434b-a5a9-897ae6ba6d33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:08.323895 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2843c79-676e-434b-a5a9-897ae6ba6d33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:08.325731 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2843c79-676e-434b-a5a9-897ae6ba6d33 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:08.340506 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 560/1104] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:39:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:39:08.347024 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2843c79-676e-434b-a5a9-897ae6ba6d33 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:08.355469 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90193e5e-ac16-4161-bfa7-7996f8501057 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:08.368913 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:08.394865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a36d30-5b95-420e-9c70-9b3450252b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:08.395518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a36d30-5b95-420e-9c70-9b3450252b36 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:08.395790 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a36d30-5b95-420e-9c70-9b3450252b36 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:08.396108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a36d30-5b95-420e-9c70-9b3450252b36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:08.396804 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a36d30-5b95-420e-9c70-9b3450252b36 None admin] RBAC: Adding query filter params (role.id=18c3c8a821b647f7982a955880bd0a9a, scope.project.id=bb116dbc16b74908bd04d1ae68630acf, user.id=de5add03e9eb46e1ba04ce0121d21dd7) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:08.397115 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a36d30-5b95-420e-9c70-9b3450252b36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:08.398620 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a36d30-5b95-420e-9c70-9b3450252b36 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:08.404682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 545/1105] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:39:08 2022] GET /identity/v3/role_assignments?user.id=de5add03e9eb46e1ba04ce0121d21dd7&scope.project.id=bb116dbc16b74908bd04d1ae68630acf&role.id=18c3c8a821b647f7982a955880bd0a9a => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:39:08.416952 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:08.458278 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2843c79-676e-434b-a5a9-897ae6ba6d33 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:08.498814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed56b5c-58d5-4564-b41b-c1ba00ac978b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:08.499501 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed56b5c-58d5-4564-b41b-c1ba00ac978b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:08.499777 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed56b5c-58d5-4564-b41b-c1ba00ac978b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:08.500047 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed56b5c-58d5-4564-b41b-c1ba00ac978b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:08.500867 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed56b5c-58d5-4564-b41b-c1ba00ac978b None admin] RBAC: Adding query filter params (role.id=f795ff29508a4109be2917083c5ead38, scope.project.id=543d44293d894a7fbb83401f7a26dd9b, user.id=f16d1768715a41648af3fe6afadd7477) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:08.501117 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed56b5c-58d5-4564-b41b-c1ba00ac978b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:08.502583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed56b5c-58d5-4564-b41b-c1ba00ac978b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:08.509638 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 561/1106] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:39:08 2022] GET /identity/v3/role_assignments?user.id=f16d1768715a41648af3fe6afadd7477&scope.project.id=543d44293d894a7fbb83401f7a26dd9b&role.id=f795ff29508a4109be2917083c5ead38 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:39:10.365860 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a62b7-3393-4b76-a668-5d01ce71ea67 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:10.366445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a62b7-3393-4b76-a668-5d01ce71ea67 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:10.366445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a62b7-3393-4b76-a668-5d01ce71ea67 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:10.368049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 546/1107] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:39:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:39:10.376822 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e93e3-a849-4a73-b116-ea91da05b82a None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:10.377449 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e93e3-a849-4a73-b116-ea91da05b82a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:10.377608 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e93e3-a849-4a73-b116-ea91da05b82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:10.400068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-671e93e3-a849-4a73-b116-ea91da05b82a None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:39:10.401347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-671e93e3-a849-4a73-b116-ea91da05b82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:39:10.482923 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 562/1108] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:39:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:39:10.762821 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a62b7-3393-4b76-a668-5d01ce71ea67 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:10.871425 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a62b7-3393-4b76-a668-5d01ce71ea67 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:10.911057 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d559da7-5507-4f4b-84ad-ee0b8e1b900c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:10.911698 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d559da7-5507-4f4b-84ad-ee0b8e1b900c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:10.911970 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d559da7-5507-4f4b-84ad-ee0b8e1b900c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:10.912243 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d559da7-5507-4f4b-84ad-ee0b8e1b900c None admin] PATH_INFO: `/v3/domains/default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:10.913531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d559da7-5507-4f4b-84ad-ee0b8e1b900c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:10.915014 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d559da7-5507-4f4b-84ad-ee0b8e1b900c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:10.916634 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 547/1109] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:39:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:10.923874 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e93e3-a849-4a73-b116-ea91da05b82a None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:10.998373 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:11.044564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e93e3-a849-4a73-b116-ea91da05b82a None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:11.082516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993f99f9-84f0-4f70-8c8c-2b6c65c9fd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:11.083176 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f99f9-84f0-4f70-8c8c-2b6c65c9fd18 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:11.083455 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f99f9-84f0-4f70-8c8c-2b6c65c9fd18 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:11.083723 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f99f9-84f0-4f70-8c8c-2b6c65c9fd18 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:11.084374 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993f99f9-84f0-4f70-8c8c-2b6c65c9fd18 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:11.085877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993f99f9-84f0-4f70-8c8c-2b6c65c9fd18 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:11.108719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 563/1110] 10.222.0.71 () {60 vars in 1132 bytes} [Fri Aug 26 08:39:10 2022] POST /identity/v3/users => generated 286 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:39:13.067265 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79185262-b956-4b67-904f-e7fb672965d2 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:13.067865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79185262-b956-4b67-904f-e7fb672965d2 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:13.067865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79185262-b956-4b67-904f-e7fb672965d2 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:13.069223 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 548/1111] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:39:13.075867 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d8534-943c-4cf3-bf32-823079e21397 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:13.076572 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d8534-943c-4cf3-bf32-823079e21397 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:13.076732 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d8534-943c-4cf3-bf32-823079e21397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:13.100811 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-577d8534-943c-4cf3-bf32-823079e21397 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:39:13.102033 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-577d8534-943c-4cf3-bf32-823079e21397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:39:13.175739 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 564/1112] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:39:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:39:13.450113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79185262-b956-4b67-904f-e7fb672965d2 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:13.503174 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:13.584851 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79185262-b956-4b67-904f-e7fb672965d2 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:13.626966 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0464f3-d663-4f64-9bd7-6e3c6d2b75c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:13.627477 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0464f3-d663-4f64-9bd7-6e3c6d2b75c1 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:13.627769 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0464f3-d663-4f64-9bd7-6e3c6d2b75c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:13.628030 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0464f3-d663-4f64-9bd7-6e3c6d2b75c1 None admin] PATH_INFO: `/v3/roles/6195dc21ae4749daa31797d2c42e050e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:13.629462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0464f3-d663-4f64-9bd7-6e3c6d2b75c1 None admin] RBAC: Authorizing `identity:get_role(role_id=6195dc21ae4749daa31797d2c42e050e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:13.630868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0464f3-d663-4f64-9bd7-6e3c6d2b75c1 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:13.632323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 549/1113] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:39:13 2022] GET /identity/v3/roles/6195dc21ae4749daa31797d2c42e050e => generated 223 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:13.638501 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d8534-943c-4cf3-bf32-823079e21397 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:13.692413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:13.756514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d8534-943c-4cf3-bf32-823079e21397 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:13.806879 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb3761-ae45-4884-b8a1-0a88046e6d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:13.807611 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb3761-ae45-4884-b8a1-0a88046e6d4b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:13.807886 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb3761-ae45-4884-b8a1-0a88046e6d4b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:13.808132 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb3761-ae45-4884-b8a1-0a88046e6d4b None admin] PATH_INFO: `/v3/users/32f1b42214be4ba4b17e65e7fe1967bd` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:13.825217 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb3761-ae45-4884-b8a1-0a88046e6d4b None admin] RBAC: Authorizing `identity:get_user(user_id=32f1b42214be4ba4b17e65e7fe1967bd)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:13.826696 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb3761-ae45-4884-b8a1-0a88046e6d4b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:13.828458 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 565/1114] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:39:13 2022] GET /identity/v3/users/32f1b42214be4ba4b17e65e7fe1967bd => generated 286 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:13.835805 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0464f3-d663-4f64-9bd7-6e3c6d2b75c1 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:13.948395 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0464f3-d663-4f64-9bd7-6e3c6d2b75c1 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:13.964113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:13.991852 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890e508-6777-4f24-ad3e-250c335aa92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:13.992491 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890e508-6777-4f24-ad3e-250c335aa92f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:13.992770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890e508-6777-4f24-ad3e-250c335aa92f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:13.993038 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890e508-6777-4f24-ad3e-250c335aa92f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:14.070829 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0890e508-6777-4f24-ad3e-250c335aa92f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:14.072360 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0890e508-6777-4f24-ad3e-250c335aa92f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:14.187829 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 550/1115] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:39:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 08:39:14.196593 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb3761-ae45-4884-b8a1-0a88046e6d4b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:14.311277 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb3761-ae45-4884-b8a1-0a88046e6d4b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:14.318983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:14.347617 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8feffb-717b-44f7-8ba6-4067be20f4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:14.348264 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8feffb-717b-44f7-8ba6-4067be20f4f1 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:14.348539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8feffb-717b-44f7-8ba6-4067be20f4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:14.348822 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8feffb-717b-44f7-8ba6-4067be20f4f1 None admin] PATH_INFO: `/v3/projects/bb116dbc16b74908bd04d1ae68630acf` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:14.349986 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8feffb-717b-44f7-8ba6-4067be20f4f1 None admin] RBAC: Authorizing `identity:get_project(project_id=bb116dbc16b74908bd04d1ae68630acf)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:14.351577 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8feffb-717b-44f7-8ba6-4067be20f4f1 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:14.353255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 566/1116] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:39:14 2022] GET /identity/v3/projects/bb116dbc16b74908bd04d1ae68630acf => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:14.359802 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890e508-6777-4f24-ad3e-250c335aa92f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:14.372661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:14.467661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890e508-6777-4f24-ad3e-250c335aa92f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:14.504403 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7b7dd-8644-4739-bcba-9c9185e7a3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:14.505016 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7b7dd-8644-4739-bcba-9c9185e7a3eb None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:14.505291 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7b7dd-8644-4739-bcba-9c9185e7a3eb None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:14.505558 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7b7dd-8644-4739-bcba-9c9185e7a3eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:14.506296 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a7b7dd-8644-4739-bcba-9c9185e7a3eb None admin] RBAC: Adding query filter params (role.id=6195dc21ae4749daa31797d2c42e050e, scope.project.id=bb116dbc16b74908bd04d1ae68630acf, user.id=32f1b42214be4ba4b17e65e7fe1967bd) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:14.506603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a7b7dd-8644-4739-bcba-9c9185e7a3eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:14.508143 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a7b7dd-8644-4739-bcba-9c9185e7a3eb None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:14.514132 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 551/1117] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:39:14 2022] GET /identity/v3/role_assignments?user.id=32f1b42214be4ba4b17e65e7fe1967bd&scope.project.id=bb116dbc16b74908bd04d1ae68630acf&role.id=6195dc21ae4749daa31797d2c42e050e => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:16.416577 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c785c-298c-4f83-8133-00f379c9db05 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:16.417149 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c785c-298c-4f83-8133-00f379c9db05 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:16.417581 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0c785c-298c-4f83-8133-00f379c9db05 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:16.418990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 567/1118] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:39:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:39:16.424698 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71bf5a-d784-4eef-9241-963b76ae5716 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:16.425173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71bf5a-d784-4eef-9241-963b76ae5716 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:16.425262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71bf5a-d784-4eef-9241-963b76ae5716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:16.444806 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ea71bf5a-d784-4eef-9241-963b76ae5716 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:39:16.446140 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ea71bf5a-d784-4eef-9241-963b76ae5716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:39:16.527162 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 552/1119] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:39:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:39:16.794170 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c785c-298c-4f83-8133-00f379c9db05 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:16.902618 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0c785c-298c-4f83-8133-00f379c9db05 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:16.939092 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f59538-ada8-42db-8384-e1ebc228ab12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:16.939755 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f59538-ada8-42db-8384-e1ebc228ab12 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:16.940027 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f59538-ada8-42db-8384-e1ebc228ab12 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:16.940290 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f59538-ada8-42db-8384-e1ebc228ab12 None admin] PATH_INFO: `/v3/roles/6195dc21ae4749daa31797d2c42e050e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:16.941700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f59538-ada8-42db-8384-e1ebc228ab12 None admin] RBAC: Authorizing `identity:get_role(role_id=6195dc21ae4749daa31797d2c42e050e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:16.943124 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f59538-ada8-42db-8384-e1ebc228ab12 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:16.944527 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 568/1120] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:39:16 2022] GET /identity/v3/roles/6195dc21ae4749daa31797d2c42e050e => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:16.950766 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71bf5a-d784-4eef-9241-963b76ae5716 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:16.954372 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:17.059109 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71bf5a-d784-4eef-9241-963b76ae5716 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:17.095273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a18cb-fe33-4824-a476-d91868c11048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:17.095944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a18cb-fe33-4824-a476-d91868c11048 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:17.096213 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a18cb-fe33-4824-a476-d91868c11048 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:17.096484 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a18cb-fe33-4824-a476-d91868c11048 None admin] PATH_INFO: `/v3/users/32f1b42214be4ba4b17e65e7fe1967bd` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:17.097727 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a18cb-fe33-4824-a476-d91868c11048 None admin] RBAC: Authorizing `identity:get_user(user_id=32f1b42214be4ba4b17e65e7fe1967bd)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:17.099220 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a18cb-fe33-4824-a476-d91868c11048 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:17.100892 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 553/1121] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:39:16 2022] GET /identity/v3/users/32f1b42214be4ba4b17e65e7fe1967bd => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:17.107387 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f59538-ada8-42db-8384-e1ebc228ab12 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:17.157819 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:17.219447 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f59538-ada8-42db-8384-e1ebc228ab12 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:17.256500 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efefb91d-ca8b-4b0c-85d3-ad83324ddc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:17.257140 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefb91d-ca8b-4b0c-85d3-ad83324ddc98 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:17.257484 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefb91d-ca8b-4b0c-85d3-ad83324ddc98 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:17.257758 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efefb91d-ca8b-4b0c-85d3-ad83324ddc98 None admin] PATH_INFO: `/v3/projects/bb116dbc16b74908bd04d1ae68630acf` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:17.258947 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efefb91d-ca8b-4b0c-85d3-ad83324ddc98 None admin] RBAC: Authorizing `identity:get_project(project_id=bb116dbc16b74908bd04d1ae68630acf)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:17.260449 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efefb91d-ca8b-4b0c-85d3-ad83324ddc98 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:17.262144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 569/1122] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:39:17 2022] GET /identity/v3/projects/bb116dbc16b74908bd04d1ae68630acf => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:17.268549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a18cb-fe33-4824-a476-d91868c11048 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:17.347810 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:17.382226 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a18cb-fe33-4824-a476-d91868c11048 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:17.419601 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5837b6da-388c-48c2-bdd0-c3b59d2f12c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:17.420261 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837b6da-388c-48c2-bdd0-c3b59d2f12c8 None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:17.420538 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837b6da-388c-48c2-bdd0-c3b59d2f12c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:17.420809 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837b6da-388c-48c2-bdd0-c3b59d2f12c8 None admin] PATH_INFO: `/v3/projects/bb116dbc16b74908bd04d1ae68630acf/users/32f1b42214be4ba4b17e65e7fe1967bd/roles/6195dc21ae4749daa31797d2c42e050e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:17.423143 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5837b6da-388c-48c2-bdd0-c3b59d2f12c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb116dbc16b74908bd04d1ae68630acf, user_id=32f1b42214be4ba4b17e65e7fe1967bd, role_id=6195dc21ae4749daa31797d2c42e050e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:17.424585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5837b6da-388c-48c2-bdd0-c3b59d2f12c8 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:17.439637 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 554/1123] 10.222.0.71 () {58 vars in 1439 bytes} [Fri Aug 26 08:39:17 2022] PUT /identity/v3/projects/bb116dbc16b74908bd04d1ae68630acf/users/32f1b42214be4ba4b17e65e7fe1967bd/roles/6195dc21ae4749daa31797d2c42e050e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:39:19.390916 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481d69b-0dec-4c8f-9a2b-ae93d552ee1d None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:19.391548 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481d69b-0dec-4c8f-9a2b-ae93d552ee1d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:19.391639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3481d69b-0dec-4c8f-9a2b-ae93d552ee1d None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:19.393515 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 570/1124] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:39:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:39:19.399755 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb864e1-5307-4726-87c2-3663401ddbef None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:19.400243 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb864e1-5307-4726-87c2-3663401ddbef None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:19.400334 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb864e1-5307-4726-87c2-3663401ddbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:19.422212 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ffb864e1-5307-4726-87c2-3663401ddbef None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:39:19.423466 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ffb864e1-5307-4726-87c2-3663401ddbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:39:19.503276 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 555/1125] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:39:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:39:19.766441 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481d69b-0dec-4c8f-9a2b-ae93d552ee1d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:19.882128 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3481d69b-0dec-4c8f-9a2b-ae93d552ee1d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:19.893049 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:19.920228 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb91599-655e-4f8b-9dee-e197a0d502bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:19.920887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb91599-655e-4f8b-9dee-e197a0d502bb None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:19.921164 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb91599-655e-4f8b-9dee-e197a0d502bb None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:19.921438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb91599-655e-4f8b-9dee-e197a0d502bb None admin] PATH_INFO: `/v3/roles/6195dc21ae4749daa31797d2c42e050e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:19.922915 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb91599-655e-4f8b-9dee-e197a0d502bb None admin] RBAC: Authorizing `identity:get_role(role_id=6195dc21ae4749daa31797d2c42e050e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:19.924331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb91599-655e-4f8b-9dee-e197a0d502bb None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:19.925689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 571/1126] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:39:19 2022] GET /identity/v3/roles/6195dc21ae4749daa31797d2c42e050e => generated 223 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:19.932128 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb864e1-5307-4726-87c2-3663401ddbef None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:20.001287 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:20.049827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb864e1-5307-4726-87c2-3663401ddbef None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:20.087230 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1add5-051d-47ab-8315-65f07a99d836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:20.087866 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1add5-051d-47ab-8315-65f07a99d836 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:20.088142 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1add5-051d-47ab-8315-65f07a99d836 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:20.088404 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1add5-051d-47ab-8315-65f07a99d836 None admin] PATH_INFO: `/v3/users/32f1b42214be4ba4b17e65e7fe1967bd` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:20.089574 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c1add5-051d-47ab-8315-65f07a99d836 None admin] RBAC: Authorizing `identity:get_user(user_id=32f1b42214be4ba4b17e65e7fe1967bd)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:20.091027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c1add5-051d-47ab-8315-65f07a99d836 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:20.092660 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 556/1127] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:39:19 2022] GET /identity/v3/users/32f1b42214be4ba4b17e65e7fe1967bd => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:20.101360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb91599-655e-4f8b-9dee-e197a0d502bb None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:20.225929 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb91599-655e-4f8b-9dee-e197a0d502bb None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:20.263350 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f871290-e6a2-4b1f-98b9-48ac0b0e3c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:20.263974 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f871290-e6a2-4b1f-98b9-48ac0b0e3c9c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:20.264235 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f871290-e6a2-4b1f-98b9-48ac0b0e3c9c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:20.264508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f871290-e6a2-4b1f-98b9-48ac0b0e3c9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:20.317981 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:20.342437 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f871290-e6a2-4b1f-98b9-48ac0b0e3c9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:20.343965 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f871290-e6a2-4b1f-98b9-48ac0b0e3c9c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:20.449899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 572/1128] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:39:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 08:39:20.456104 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1add5-051d-47ab-8315-65f07a99d836 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:20.565034 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1add5-051d-47ab-8315-65f07a99d836 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:20.596606 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:20.602721 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da0fa3-7b65-445a-a363-17c46f49e4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:20.603542 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da0fa3-7b65-445a-a363-17c46f49e4d7 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:20.603813 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da0fa3-7b65-445a-a363-17c46f49e4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:20.604079 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da0fa3-7b65-445a-a363-17c46f49e4d7 None admin] PATH_INFO: `/v3/projects/bb116dbc16b74908bd04d1ae68630acf` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:20.605199 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da0fa3-7b65-445a-a363-17c46f49e4d7 None admin] RBAC: Authorizing `identity:get_project(project_id=bb116dbc16b74908bd04d1ae68630acf)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:20.606652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da0fa3-7b65-445a-a363-17c46f49e4d7 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:20.608389 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 557/1129] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:39:20 2022] GET /identity/v3/projects/bb116dbc16b74908bd04d1ae68630acf => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:20.615493 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f871290-e6a2-4b1f-98b9-48ac0b0e3c9c None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:20.698928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:20.724799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f871290-e6a2-4b1f-98b9-48ac0b0e3c9c None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:20.770874 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5da80-e056-4fe8-a8d5-9336f97484fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:20.771606 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5da80-e056-4fe8-a8d5-9336f97484fe None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:20.771888 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5da80-e056-4fe8-a8d5-9336f97484fe None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:20.772156 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5da80-e056-4fe8-a8d5-9336f97484fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:20.772898 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a5da80-e056-4fe8-a8d5-9336f97484fe None admin] RBAC: Adding query filter params (role.id=6195dc21ae4749daa31797d2c42e050e, scope.project.id=bb116dbc16b74908bd04d1ae68630acf, user.id=32f1b42214be4ba4b17e65e7fe1967bd) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:20.773254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a5da80-e056-4fe8-a8d5-9336f97484fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:20.774700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a5da80-e056-4fe8-a8d5-9336f97484fe None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:20.781241 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 573/1130] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:39:20 2022] GET /identity/v3/role_assignments?user.id=32f1b42214be4ba4b17e65e7fe1967bd&scope.project.id=bb116dbc16b74908bd04d1ae68630acf&role.id=6195dc21ae4749daa31797d2c42e050e => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:22.712440 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6252c1-a448-486e-bad3-de3628cc38fd None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:22.713095 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6252c1-a448-486e-bad3-de3628cc38fd None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:22.713095 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6252c1-a448-486e-bad3-de3628cc38fd None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:22.714379 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 558/1131] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:39:22.720283 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c630f-9780-4014-9f55-34e1133c32f3 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:22.720791 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c630f-9780-4014-9f55-34e1133c32f3 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:22.720791 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c630f-9780-4014-9f55-34e1133c32f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:22.740974 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-232c630f-9780-4014-9f55-34e1133c32f3 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:39:22.742186 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-232c630f-9780-4014-9f55-34e1133c32f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:39:22.816946 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 574/1132] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:39:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:39:23.078872 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6252c1-a448-486e-bad3-de3628cc38fd None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:23.191829 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6252c1-a448-486e-bad3-de3628cc38fd None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:23.233595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7796e1-b8cc-49ef-9b3a-026ee331e0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:23.234227 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7796e1-b8cc-49ef-9b3a-026ee331e0cb None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:23.234503 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7796e1-b8cc-49ef-9b3a-026ee331e0cb None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:23.234771 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7796e1-b8cc-49ef-9b3a-026ee331e0cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:23.236051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7796e1-b8cc-49ef-9b3a-026ee331e0cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:23.237493 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7796e1-b8cc-49ef-9b3a-026ee331e0cb None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:23.239130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 559/1133] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:39:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:23.248338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c630f-9780-4014-9f55-34e1133c32f3 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:23.325387 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:23.362295 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c630f-9780-4014-9f55-34e1133c32f3 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:23.400119 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e257d8-38d3-4d56-b42e-10b89913d6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:23.400729 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e257d8-38d3-4d56-b42e-10b89913d6d2 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:23.401004 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e257d8-38d3-4d56-b42e-10b89913d6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:23.401270 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e257d8-38d3-4d56-b42e-10b89913d6d2 None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:23.401921 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e257d8-38d3-4d56-b42e-10b89913d6d2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:23.403406 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e257d8-38d3-4d56-b42e-10b89913d6d2 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:23.440830 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 575/1134] 10.222.0.71 () {60 vars in 1141 bytes} [Fri Aug 26 08:39:23 2022] POST /identity/v3/projects => generated 311 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:39:25.370684 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4d1bb-1054-4e92-9d6b-52bce0edc971 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:25.371288 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4d1bb-1054-4e92-9d6b-52bce0edc971 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:25.371288 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4d1bb-1054-4e92-9d6b-52bce0edc971 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:25.373009 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 560/1135] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:39:25.379563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301f55e-bf36-4ad3-a81f-a835f561c1d3 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:25.380057 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301f55e-bf36-4ad3-a81f-a835f561c1d3 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:25.380057 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301f55e-bf36-4ad3-a81f-a835f561c1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:25.399864 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-3301f55e-bf36-4ad3-a81f-a835f561c1d3 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:39:25.401072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3301f55e-bf36-4ad3-a81f-a835f561c1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:39:25.472131 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 576/1136] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:39:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:39:25.736127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4d1bb-1054-4e92-9d6b-52bce0edc971 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:25.809417 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:25.846814 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4d1bb-1054-4e92-9d6b-52bce0edc971 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:25.883426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3d4f4-cab8-41d3-b2c0-7d220bc1e6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:25.884055 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3d4f4-cab8-41d3-b2c0-7d220bc1e6d3 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:25.884339 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3d4f4-cab8-41d3-b2c0-7d220bc1e6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:25.884609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3d4f4-cab8-41d3-b2c0-7d220bc1e6d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:25.885785 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f3d4f4-cab8-41d3-b2c0-7d220bc1e6d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:25.887257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f3d4f4-cab8-41d3-b2c0-7d220bc1e6d3 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:25.888864 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 561/1137] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:39:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:25.894914 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301f55e-bf36-4ad3-a81f-a835f561c1d3 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:25.953808 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:26.004724 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301f55e-bf36-4ad3-a81f-a835f561c1d3 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:26.048679 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e78c46-e87f-4d30-8615-ef0f7091ba64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:26.049300 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e78c46-e87f-4d30-8615-ef0f7091ba64 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:26.049574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e78c46-e87f-4d30-8615-ef0f7091ba64 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:26.049841 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e78c46-e87f-4d30-8615-ef0f7091ba64 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:26.050507 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e78c46-e87f-4d30-8615-ef0f7091ba64 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:26.051993 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e78c46-e87f-4d30-8615-ef0f7091ba64 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:26.074593 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 577/1138] 10.222.0.71 () {60 vars in 1132 bytes} [Fri Aug 26 08:39:25 2022] POST /identity/v3/users => generated 286 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:39:28.031383 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4f2af-372c-442d-bf96-9fad994ba1e6 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:28.032360 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4f2af-372c-442d-bf96-9fad994ba1e6 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:28.032360 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4f2af-372c-442d-bf96-9fad994ba1e6 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:28.034364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 562/1139] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:39:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:39:28.042227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9068ad-f7e7-4a05-80e6-2d3e5f2c4853 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:28.042768 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9068ad-f7e7-4a05-80e6-2d3e5f2c4853 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:28.042768 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9068ad-f7e7-4a05-80e6-2d3e5f2c4853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:28.063722 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-3c9068ad-f7e7-4a05-80e6-2d3e5f2c4853 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:39:28.064904 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3c9068ad-f7e7-4a05-80e6-2d3e5f2c4853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:39:28.147195 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 578/1140] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:39:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:39:28.408824 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4f2af-372c-442d-bf96-9fad994ba1e6 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:28.517784 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4f2af-372c-442d-bf96-9fad994ba1e6 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:28.557767 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1f117-3a4d-4e6e-ba51-c9873014cef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:28.558411 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1f117-3a4d-4e6e-ba51-c9873014cef5 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:28.558683 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1f117-3a4d-4e6e-ba51-c9873014cef5 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:28.558981 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1f117-3a4d-4e6e-ba51-c9873014cef5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:28.561096 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:28.572944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd1f117-3a4d-4e6e-ba51-c9873014cef5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:39:28.577616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-0bd1f117-3a4d-4e6e-ba51-c9873014cef5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:39:28.577616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:39:28.577616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:39:28.577616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:39:28.577616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:39:28.577616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:39:28.577616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:39:28.577616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:39:28.577616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:39:28.577616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:39:28.577616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:39:28.577616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:39:28.577616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:39:28.577616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:39:28.577616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:39:28.577616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:39:28.578796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:39:28.578796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:39:28.578796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:39:28.578796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:39:28.578796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:39:28.578796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:39:28.578796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:39:28.578796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:39:28.578796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:39:28.578796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:39:28.578796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:39:28.578796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:39:28.578796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:39:28.578796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:39:28.578796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:39:28.578796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:39:28.579735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:39:28.579735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:39:28.579735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:39:28.579735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:39:28.579735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:39:28.579735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:39:28.579735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:39:28.579735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:39:28.579735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:39:28.579735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:39:28.579735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:39:28.579735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:39:28.579735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 563/1141] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:39:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:39:28.585055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9068ad-f7e7-4a05-80e6-2d3e5f2c4853 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:28.647384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:28.720122 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9068ad-f7e7-4a05-80e6-2d3e5f2c4853 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:28.761876 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cb6b8-794f-4fdd-b653-00805be9f610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:28.762543 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cb6b8-794f-4fdd-b653-00805be9f610 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:28.762795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cb6b8-794f-4fdd-b653-00805be9f610 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:28.763087 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cb6b8-794f-4fdd-b653-00805be9f610 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:28.763776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033cb6b8-794f-4fdd-b653-00805be9f610 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:28.764034 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033cb6b8-794f-4fdd-b653-00805be9f610 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:28.765440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033cb6b8-794f-4fdd-b653-00805be9f610 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:28.778228 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 579/1142] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:39:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:28.784411 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1f117-3a4d-4e6e-ba51-c9873014cef5 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:28.919063 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1f117-3a4d-4e6e-ba51-c9873014cef5 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:28.953603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2243290-e1ae-483f-a2db-46e4459e0256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:28.954226 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2243290-e1ae-483f-a2db-46e4459e0256 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:28.954501 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2243290-e1ae-483f-a2db-46e4459e0256 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:28.954768 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2243290-e1ae-483f-a2db-46e4459e0256 None admin] PATH_INFO: `/v3/users/c9e112beb4ac4c65ba8ff4d7c4c0bc0d` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:28.968662 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2243290-e1ae-483f-a2db-46e4459e0256 None admin] RBAC: Authorizing `identity:get_user(user_id=c9e112beb4ac4c65ba8ff4d7c4c0bc0d)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:28.970118 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2243290-e1ae-483f-a2db-46e4459e0256 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:28.971873 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 564/1143] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:39:28 2022] GET /identity/v3/users/c9e112beb4ac4c65ba8ff4d7c4c0bc0d => generated 286 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:28.978277 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cb6b8-794f-4fdd-b653-00805be9f610 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:29.084545 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cb6b8-794f-4fdd-b653-00805be9f610 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:29.086437 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:29.124949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454b6ef-75e0-4a5d-8385-faa5cf730dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:29.125735 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454b6ef-75e0-4a5d-8385-faa5cf730dcd None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:29.125991 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454b6ef-75e0-4a5d-8385-faa5cf730dcd None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:29.126087 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454b6ef-75e0-4a5d-8385-faa5cf730dcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:29.198823 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454b6ef-75e0-4a5d-8385-faa5cf730dcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:29.200254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454b6ef-75e0-4a5d-8385-faa5cf730dcd None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:29.334468 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 580/1144] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:39:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 08:39:29.340959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2243290-e1ae-483f-a2db-46e4459e0256 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:29.403715 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:29.453347 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2243290-e1ae-483f-a2db-46e4459e0256 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:29.499880 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e498a9-c082-40db-a13f-c3ba8e1fadef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:29.500554 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e498a9-c082-40db-a13f-c3ba8e1fadef None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:29.500832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e498a9-c082-40db-a13f-c3ba8e1fadef None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:29.501107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e498a9-c082-40db-a13f-c3ba8e1fadef None admin] PATH_INFO: `/v3/projects/fcba399164724e5e86054ca4b5be66fe` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:29.502574 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e498a9-c082-40db-a13f-c3ba8e1fadef None admin] RBAC: Authorizing `identity:get_project(project_id=fcba399164724e5e86054ca4b5be66fe)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:29.504099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e498a9-c082-40db-a13f-c3ba8e1fadef None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:29.505802 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 565/1145] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:39:29 2022] GET /identity/v3/projects/fcba399164724e5e86054ca4b5be66fe => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:29.512775 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454b6ef-75e0-4a5d-8385-faa5cf730dcd None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:29.515135 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:29.642754 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454b6ef-75e0-4a5d-8385-faa5cf730dcd None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:29.679936 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b8a0f5-16bf-4540-be12-2c31ec099940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:29.680548 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8a0f5-16bf-4540-be12-2c31ec099940 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:29.680824 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8a0f5-16bf-4540-be12-2c31ec099940 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:29.681090 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b8a0f5-16bf-4540-be12-2c31ec099940 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:29.681814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8a0f5-16bf-4540-be12-2c31ec099940 None admin] RBAC: Adding query filter params (role.id=18c3c8a821b647f7982a955880bd0a9a, scope.project.id=fcba399164724e5e86054ca4b5be66fe, user.id=c9e112beb4ac4c65ba8ff4d7c4c0bc0d) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:29.682119 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8a0f5-16bf-4540-be12-2c31ec099940 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:29.683589 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b8a0f5-16bf-4540-be12-2c31ec099940 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:29.689148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 581/1146] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:39:29 2022] GET /identity/v3/role_assignments?user.id=c9e112beb4ac4c65ba8ff4d7c4c0bc0d&scope.project.id=fcba399164724e5e86054ca4b5be66fe&role.id=18c3c8a821b647f7982a955880bd0a9a => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:31.563151 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9f0a1-29a2-4821-b821-0833975b896a None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:31.563749 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9f0a1-29a2-4821-b821-0833975b896a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:31.563749 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9f0a1-29a2-4821-b821-0833975b896a None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:31.565047 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 566/1147] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:39:31.571123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc844e-d3e0-4938-8b83-aa72bd247bdd None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:31.572033 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc844e-d3e0-4938-8b83-aa72bd247bdd None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:31.572033 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc844e-d3e0-4938-8b83-aa72bd247bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:31.591453 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-5efc844e-d3e0-4938-8b83-aa72bd247bdd None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:39:31.592711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-5efc844e-d3e0-4938-8b83-aa72bd247bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:39:31.670369 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:31.679081 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 582/1148] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:39:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:39:31.942210 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9f0a1-29a2-4821-b821-0833975b896a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:32.060309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9f0a1-29a2-4821-b821-0833975b896a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:32.081496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:32.096371 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04077f83-e0ea-4e7f-b7f9-dbf8b19320ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:32.097014 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04077f83-e0ea-4e7f-b7f9-dbf8b19320ab None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:32.097287 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04077f83-e0ea-4e7f-b7f9-dbf8b19320ab None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:32.097551 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04077f83-e0ea-4e7f-b7f9-dbf8b19320ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:32.107949 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04077f83-e0ea-4e7f-b7f9-dbf8b19320ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:39:32.112650 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-04077f83-e0ea-4e7f-b7f9-dbf8b19320ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:39:32.112650 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:39:32.112650 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:39:32.112650 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:39:32.112650 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:39:32.112650 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:39:32.112650 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:39:32.112650 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:39:32.112650 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:39:32.112650 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:39:32.112650 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:39:32.112650 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:39:32.112650 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:39:32.112650 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:39:32.112650 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:39:32.112650 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:39:32.113699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:39:32.113699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:39:32.113699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:39:32.113699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:39:32.113699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:39:32.113699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:39:32.113699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:39:32.113699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:39:32.113699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:39:32.113699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:39:32.113699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:39:32.113699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:39:32.113699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:39:32.113699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:39:32.113699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:39:32.113699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:39:32.114606 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:39:32.114606 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:39:32.114606 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:39:32.114606 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:39:32.114606 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:39:32.114606 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:39:32.114606 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:39:32.114606 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:39:32.114606 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:39:32.114606 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:39:32.114606 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:39:32.114606 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:39:32.114606 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 567/1149] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:39:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:39:32.121010 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc844e-d3e0-4938-8b83-aa72bd247bdd None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:32.230457 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc844e-d3e0-4938-8b83-aa72bd247bdd None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:32.270215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2816ca-98b7-4e9e-8831-521759a607a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:32.270835 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2816ca-98b7-4e9e-8831-521759a607a2 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:32.271141 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2816ca-98b7-4e9e-8831-521759a607a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:32.271398 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2816ca-98b7-4e9e-8831-521759a607a2 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:32.272068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2816ca-98b7-4e9e-8831-521759a607a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:32.272369 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2816ca-98b7-4e9e-8831-521759a607a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:32.273896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2816ca-98b7-4e9e-8831-521759a607a2 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:32.283231 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 583/1150] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:39:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:32.290822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04077f83-e0ea-4e7f-b7f9-dbf8b19320ab None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:32.424645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04077f83-e0ea-4e7f-b7f9-dbf8b19320ab None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:32.462668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322687a5-15c0-4c2a-b3eb-bb0c9bd17f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:32.463324 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322687a5-15c0-4c2a-b3eb-bb0c9bd17f2c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:32.463603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322687a5-15c0-4c2a-b3eb-bb0c9bd17f2c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:32.463876 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322687a5-15c0-4c2a-b3eb-bb0c9bd17f2c None admin] PATH_INFO: `/v3/users/c9e112beb4ac4c65ba8ff4d7c4c0bc0d` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:32.465111 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322687a5-15c0-4c2a-b3eb-bb0c9bd17f2c None admin] RBAC: Authorizing `identity:get_user(user_id=c9e112beb4ac4c65ba8ff4d7c4c0bc0d)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:32.466567 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322687a5-15c0-4c2a-b3eb-bb0c9bd17f2c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:32.468258 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 568/1151] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:39:32 2022] GET /identity/v3/users/c9e112beb4ac4c65ba8ff4d7c4c0bc0d => generated 286 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:32.474405 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2816ca-98b7-4e9e-8831-521759a607a2 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:32.538735 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:32.587079 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2816ca-98b7-4e9e-8831-521759a607a2 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:32.626683 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59381ca-12ee-4485-bb7f-90773e2319b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:32.627375 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59381ca-12ee-4485-bb7f-90773e2319b6 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:32.627652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59381ca-12ee-4485-bb7f-90773e2319b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:32.627923 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59381ca-12ee-4485-bb7f-90773e2319b6 None admin] PATH_INFO: `/v3/projects/fcba399164724e5e86054ca4b5be66fe` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:32.629126 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59381ca-12ee-4485-bb7f-90773e2319b6 None admin] RBAC: Authorizing `identity:get_project(project_id=fcba399164724e5e86054ca4b5be66fe)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:32.630626 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59381ca-12ee-4485-bb7f-90773e2319b6 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:32.638030 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 584/1152] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:39:32 2022] GET /identity/v3/projects/fcba399164724e5e86054ca4b5be66fe => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:32.639470 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322687a5-15c0-4c2a-b3eb-bb0c9bd17f2c None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:32.696332 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:32.759033 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322687a5-15c0-4c2a-b3eb-bb0c9bd17f2c None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:32.798400 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9889d3-a82d-40ee-8421-36af3febabfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:32.798993 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9889d3-a82d-40ee-8421-36af3febabfc None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:32.799272 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9889d3-a82d-40ee-8421-36af3febabfc None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:32.799541 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9889d3-a82d-40ee-8421-36af3febabfc None admin] PATH_INFO: `/v3/projects/fcba399164724e5e86054ca4b5be66fe/users/c9e112beb4ac4c65ba8ff4d7c4c0bc0d/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:32.801526 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9889d3-a82d-40ee-8421-36af3febabfc None admin] RBAC: Authorizing `identity:create_grant(project_id=fcba399164724e5e86054ca4b5be66fe, user_id=c9e112beb4ac4c65ba8ff4d7c4c0bc0d, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:32.803075 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9889d3-a82d-40ee-8421-36af3febabfc None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:32.820308 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 569/1153] 10.222.0.71 () {58 vars in 1439 bytes} [Fri Aug 26 08:39:32 2022] PUT /identity/v3/projects/fcba399164724e5e86054ca4b5be66fe/users/c9e112beb4ac4c65ba8ff4d7c4c0bc0d/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:39:34.691601 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277734d-ecf0-4db5-89d0-d443dde699d4 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:34.692450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277734d-ecf0-4db5-89d0-d443dde699d4 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:34.692450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277734d-ecf0-4db5-89d0-d443dde699d4 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:34.693799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 585/1154] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:39:34.699520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f6171-b419-484e-b150-19111f6fa913 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:34.700017 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f6171-b419-484e-b150-19111f6fa913 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:34.700017 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f6171-b419-484e-b150-19111f6fa913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:34.720059 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-561f6171-b419-484e-b150-19111f6fa913 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:39:34.721258 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-561f6171-b419-484e-b150-19111f6fa913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:39:34.794857 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 570/1155] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:39:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:39:35.060376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277734d-ecf0-4db5-89d0-d443dde699d4 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:35.190248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277734d-ecf0-4db5-89d0-d443dde699d4 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:35.220090 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:35.234047 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8058b72-824a-4304-9451-927f19b85f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:35.235026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8058b72-824a-4304-9451-927f19b85f81 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:35.235415 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8058b72-824a-4304-9451-927f19b85f81 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:35.235786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8058b72-824a-4304-9451-927f19b85f81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:35.247140 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8058b72-824a-4304-9451-927f19b85f81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:39:35.252210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-d8058b72-824a-4304-9451-927f19b85f81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:39:35.252210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:39:35.252210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:39:35.252210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:39:35.252210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:39:35.252210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:39:35.252210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:39:35.252210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:39:35.252210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:39:35.252210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:39:35.252210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:39:35.252210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:39:35.252210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:39:35.252210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:39:35.252210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:39:35.252210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:39:35.253335 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:39:35.253335 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:39:35.253335 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:39:35.253335 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:39:35.253335 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:39:35.253335 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:39:35.253335 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:39:35.253335 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:39:35.253335 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:39:35.253335 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:39:35.253335 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:39:35.253335 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:39:35.253335 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:39:35.253335 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:39:35.253335 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:39:35.253335 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:39:35.254966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:39:35.254966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:39:35.254966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:39:35.254966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:39:35.254966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:39:35.254966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:39:35.254966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:39:35.254966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:39:35.254966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:39:35.254966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:39:35.254966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:39:35.254966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:39:35.254966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 586/1156] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:39:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:39:35.261743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f6171-b419-484e-b150-19111f6fa913 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:35.303412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:35.370136 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f6171-b419-484e-b150-19111f6fa913 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:35.417410 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea9065-46c3-4c33-89d9-717ee2d259a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:35.418074 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea9065-46c3-4c33-89d9-717ee2d259a2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:35.418352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea9065-46c3-4c33-89d9-717ee2d259a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:35.418636 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea9065-46c3-4c33-89d9-717ee2d259a2 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:35.419389 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eea9065-46c3-4c33-89d9-717ee2d259a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:35.419712 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eea9065-46c3-4c33-89d9-717ee2d259a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:35.421173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eea9065-46c3-4c33-89d9-717ee2d259a2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:35.431061 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 571/1157] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:39:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:35.437686 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8058b72-824a-4304-9451-927f19b85f81 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:35.554735 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8058b72-824a-4304-9451-927f19b85f81 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:35.592846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c26b4-3ea1-4d13-8ab6-eba6ff7b8493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:35.593474 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c26b4-3ea1-4d13-8ab6-eba6ff7b8493 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:35.593749 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c26b4-3ea1-4d13-8ab6-eba6ff7b8493 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:35.594020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c26b4-3ea1-4d13-8ab6-eba6ff7b8493 None admin] PATH_INFO: `/v3/users/c9e112beb4ac4c65ba8ff4d7c4c0bc0d` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:35.595226 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c26b4-3ea1-4d13-8ab6-eba6ff7b8493 None admin] RBAC: Authorizing `identity:get_user(user_id=c9e112beb4ac4c65ba8ff4d7c4c0bc0d)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:35.596673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c26b4-3ea1-4d13-8ab6-eba6ff7b8493 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:35.598310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 587/1158] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:39:35 2022] GET /identity/v3/users/c9e112beb4ac4c65ba8ff4d7c4c0bc0d => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:35.608223 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea9065-46c3-4c33-89d9-717ee2d259a2 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:35.718730 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:35.723095 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea9065-46c3-4c33-89d9-717ee2d259a2 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:35.771043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b3da3-93d7-4225-8202-5683650b1b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:35.771673 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b3da3-93d7-4225-8202-5683650b1b90 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:35.771945 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b3da3-93d7-4225-8202-5683650b1b90 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:35.772215 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b3da3-93d7-4225-8202-5683650b1b90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:35.846541 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b3da3-93d7-4225-8202-5683650b1b90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:35.848065 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b3da3-93d7-4225-8202-5683650b1b90 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:35.953738 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:35.957840 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 572/1159] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:39:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 08:39:35.964645 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c26b4-3ea1-4d13-8ab6-eba6ff7b8493 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:36.028716 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:36.082786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c26b4-3ea1-4d13-8ab6-eba6ff7b8493 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:36.121581 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907a3eb-857e-4f4a-8d28-2b2f7a9dfd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:36.122224 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907a3eb-857e-4f4a-8d28-2b2f7a9dfd8e None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:36.122509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907a3eb-857e-4f4a-8d28-2b2f7a9dfd8e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:36.122783 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907a3eb-857e-4f4a-8d28-2b2f7a9dfd8e None admin] PATH_INFO: `/v3/projects/fcba399164724e5e86054ca4b5be66fe` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:36.124309 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907a3eb-857e-4f4a-8d28-2b2f7a9dfd8e None admin] RBAC: Authorizing `identity:get_project(project_id=fcba399164724e5e86054ca4b5be66fe)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:36.125766 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907a3eb-857e-4f4a-8d28-2b2f7a9dfd8e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:36.127506 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 588/1160] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:39:35 2022] GET /identity/v3/projects/fcba399164724e5e86054ca4b5be66fe => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:36.134506 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b3da3-93d7-4225-8202-5683650b1b90 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:36.234780 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b3da3-93d7-4225-8202-5683650b1b90 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:36.271703 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86920002-8d62-4bab-8426-33744ed0d61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:36.272310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86920002-8d62-4bab-8426-33744ed0d61c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:36.272565 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86920002-8d62-4bab-8426-33744ed0d61c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:36.272863 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86920002-8d62-4bab-8426-33744ed0d61c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:36.273624 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86920002-8d62-4bab-8426-33744ed0d61c None admin] RBAC: Adding query filter params (role.id=18c3c8a821b647f7982a955880bd0a9a, scope.project.id=fcba399164724e5e86054ca4b5be66fe, user.id=c9e112beb4ac4c65ba8ff4d7c4c0bc0d) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:36.273938 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86920002-8d62-4bab-8426-33744ed0d61c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:36.275439 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86920002-8d62-4bab-8426-33744ed0d61c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:36.281670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 573/1161] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:39:36 2022] GET /identity/v3/role_assignments?user.id=c9e112beb4ac4c65ba8ff4d7c4c0bc0d&scope.project.id=fcba399164724e5e86054ca4b5be66fe&role.id=18c3c8a821b647f7982a955880bd0a9a => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:38.225986 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225560b9-69fd-4bdc-821c-fd396eab5694 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:38.226764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225560b9-69fd-4bdc-821c-fd396eab5694 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:38.226764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225560b9-69fd-4bdc-821c-fd396eab5694 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:38.229071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 589/1162] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:39:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:39:38.234601 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fec334-7ae8-4bdc-8ad1-691cd28251ce None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:38.235047 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fec334-7ae8-4bdc-8ad1-691cd28251ce None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:38.235166 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fec334-7ae8-4bdc-8ad1-691cd28251ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:38.261024 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-f0fec334-7ae8-4bdc-8ad1-691cd28251ce None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:39:38.262239 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f0fec334-7ae8-4bdc-8ad1-691cd28251ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:39:38.322903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:38.332725 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 574/1163] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:39:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:39:38.595522 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225560b9-69fd-4bdc-821c-fd396eab5694 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:38.719893 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225560b9-69fd-4bdc-821c-fd396eab5694 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:38.744878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:38.761084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56361f8b-48f5-4283-b6db-0c5a40c2523d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:38.761696 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56361f8b-48f5-4283-b6db-0c5a40c2523d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:38.761976 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56361f8b-48f5-4283-b6db-0c5a40c2523d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:38.762247 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56361f8b-48f5-4283-b6db-0c5a40c2523d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:38.833992 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56361f8b-48f5-4283-b6db-0c5a40c2523d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:38.835472 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56361f8b-48f5-4283-b6db-0c5a40c2523d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:38.937741 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 590/1164] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:39:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 08:39:38.944348 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fec334-7ae8-4bdc-8ad1-691cd28251ce None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:39.048811 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fec334-7ae8-4bdc-8ad1-691cd28251ce None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:39.084781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d476ef3-e9d1-44b8-bb0e-ea9adb6cc948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:39.085772 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d476ef3-e9d1-44b8-bb0e-ea9adb6cc948 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:39.085772 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d476ef3-e9d1-44b8-bb0e-ea9adb6cc948 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:39.085966 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d476ef3-e9d1-44b8-bb0e-ea9adb6cc948 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:39.092990 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d476ef3-e9d1-44b8-bb0e-ea9adb6cc948 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:39.094440 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d476ef3-e9d1-44b8-bb0e-ea9adb6cc948 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:39.106595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-7d476ef3-e9d1-44b8-bb0e-ea9adb6cc948 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 26 08:39:39.106595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:39:39.106595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 26 08:39:39.106595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 26 08:39:39.106595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:39:39.106595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:39:39.106595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:39:39.106595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:39:39.106595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 26 08:39:39.106595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:39:39.106595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:39:39.106595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:39:39.106595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:39:39.106595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:39:39.106595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:39:39.106595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:39:39.106595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:39:39.107822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:39:39.107822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:39:39.107822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:39:39.107822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:39:39.107822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:39:39.107822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:39:39.107822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:39:39.107822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:39:39.107822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:39:39.107822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:39:39.107822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:39:39.107822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:39:39.107822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:39:39.107822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:39:39.107822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:39:39.107822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:39:39.108825 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:39:39.108825 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:39:39.108825 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:39:39.108825 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:39:39.108825 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:39:39.108825 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:39:39.108825 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:39:39.108825 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:39:39.108825 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:39:39.108825 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:39:39.108825 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 26 08:39:39.108825 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:39:39.108825 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 26 08:39:39.108825 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:39:39.111095 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 575/1165] 10.222.0.71 () {56 vars in 1117 bytes} [Fri Aug 26 08:39:38 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:39:39.114228 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56361f8b-48f5-4283-b6db-0c5a40c2523d None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:39.142958 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:39.226027 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56361f8b-48f5-4283-b6db-0c5a40c2523d None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:39.262916 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008087c1-8134-4628-807e-98432ce9c12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:39.263624 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008087c1-8134-4628-807e-98432ce9c12f None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:39.263806 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008087c1-8134-4628-807e-98432ce9c12f None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:39.264081 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008087c1-8134-4628-807e-98432ce9c12f None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:39.264724 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008087c1-8134-4628-807e-98432ce9c12f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:39.265030 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008087c1-8134-4628-807e-98432ce9c12f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:39.266410 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008087c1-8134-4628-807e-98432ce9c12f None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:39.274096 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 591/1166] 10.222.0.71 () {56 vars in 1115 bytes} [Fri Aug 26 08:39:39 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:41.188738 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf6c86-e08b-45df-ab57-5468a149635d None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:41.189591 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf6c86-e08b-45df-ab57-5468a149635d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:41.189591 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf6c86-e08b-45df-ab57-5468a149635d None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:41.191893 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 576/1167] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:39:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:39:41.199560 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8298a1d-af36-457d-bae3-325ff696b32d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:41.200031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8298a1d-af36-457d-bae3-325ff696b32d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:41.200141 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8298a1d-af36-457d-bae3-325ff696b32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:41.221302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-f8298a1d-af36-457d-bae3-325ff696b32d None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:39:41.222558 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f8298a1d-af36-457d-bae3-325ff696b32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:39:41.298175 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 592/1168] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:39:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:39:41.564387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf6c86-e08b-45df-ab57-5468a149635d None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:41.636879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:41.690251 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf6c86-e08b-45df-ab57-5468a149635d None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:41.729354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450627a2-306d-482a-8be9-fa9c080b5950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:41.730071 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450627a2-306d-482a-8be9-fa9c080b5950 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:41.730355 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450627a2-306d-482a-8be9-fa9c080b5950 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:41.730623 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450627a2-306d-482a-8be9-fa9c080b5950 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:41.731313 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450627a2-306d-482a-8be9-fa9c080b5950 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:41.732687 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450627a2-306d-482a-8be9-fa9c080b5950 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:41.755078 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 577/1169] 10.222.0.71 () {60 vars in 1138 bytes} [Fri Aug 26 08:39:41 2022] POST /identity/v3/domains => generated 267 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:39:43.674177 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729178a6-366b-480d-9cdf-9e91ab1c3877 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:43.674715 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729178a6-366b-480d-9cdf-9e91ab1c3877 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:43.674715 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729178a6-366b-480d-9cdf-9e91ab1c3877 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:43.676188 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 593/1170] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:39:43.681548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f150a62-08e0-4ef7-a036-87e975dc3ed0 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:43.682000 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f150a62-08e0-4ef7-a036-87e975dc3ed0 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:43.682089 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f150a62-08e0-4ef7-a036-87e975dc3ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:43.702356 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-8f150a62-08e0-4ef7-a036-87e975dc3ed0 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:39:43.703963 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8f150a62-08e0-4ef7-a036-87e975dc3ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:39:43.778755 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 578/1171] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:39:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:39:44.045089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729178a6-366b-480d-9cdf-9e91ab1c3877 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:44.074757 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:44.173508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729178a6-366b-480d-9cdf-9e91ab1c3877 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:44.211541 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8dd890-7609-4d74-8d5a-e5b89396c06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:44.212241 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8dd890-7609-4d74-8d5a-e5b89396c06c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:44.212516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8dd890-7609-4d74-8d5a-e5b89396c06c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:44.212758 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8dd890-7609-4d74-8d5a-e5b89396c06c None admin] PATH_INFO: `/v3/domains/fcbd782c7f7b49158dfc3fecbb2a9596` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:44.223707 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8dd890-7609-4d74-8d5a-e5b89396c06c None admin] RBAC: Authorizing `identity:get_domain(domain_id=fcbd782c7f7b49158dfc3fecbb2a9596)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:44.225167 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8dd890-7609-4d74-8d5a-e5b89396c06c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:44.226782 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 594/1172] 10.222.0.71 () {56 vars in 1183 bytes} [Fri Aug 26 08:39:44 2022] GET /identity/v3/domains/fcbd782c7f7b49158dfc3fecbb2a9596 => generated 267 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:44.233077 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f150a62-08e0-4ef7-a036-87e975dc3ed0 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:44.281462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:44.337469 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f150a62-08e0-4ef7-a036-87e975dc3ed0 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:44.372837 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973104a-b60e-4659-83e3-c60315add03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:44.373814 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973104a-b60e-4659-83e3-c60315add03b None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:44.374086 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973104a-b60e-4659-83e3-c60315add03b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:44.374347 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973104a-b60e-4659-83e3-c60315add03b None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:44.375044 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a973104a-b60e-4659-83e3-c60315add03b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:44.376486 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a973104a-b60e-4659-83e3-c60315add03b None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:44.407842 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 579/1173] 10.222.0.71 () {60 vars in 1141 bytes} [Fri Aug 26 08:39:44 2022] POST /identity/v3/projects => generated 361 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:39:46.330851 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1598c3-bbfa-4c88-8511-96f2fd88f27d None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:46.331776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1598c3-bbfa-4c88-8511-96f2fd88f27d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:46.331776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1598c3-bbfa-4c88-8511-96f2fd88f27d None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:46.333989 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 595/1174] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:39:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:39:46.340584 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017641bb-33ea-4d6e-b46b-a86eb5c396f8 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:46.341345 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017641bb-33ea-4d6e-b46b-a86eb5c396f8 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:46.341498 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017641bb-33ea-4d6e-b46b-a86eb5c396f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:46.370507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-017641bb-33ea-4d6e-b46b-a86eb5c396f8 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:39:46.371783 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-017641bb-33ea-4d6e-b46b-a86eb5c396f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:39:46.440582 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 580/1175] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:39:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:39:46.720539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1598c3-bbfa-4c88-8511-96f2fd88f27d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:46.808226 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:46.833605 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1598c3-bbfa-4c88-8511-96f2fd88f27d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:46.869535 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ee597-2150-4d53-a529-9ef782740892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:46.870157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ee597-2150-4d53-a529-9ef782740892 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:46.870431 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ee597-2150-4d53-a529-9ef782740892 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:46.870694 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ee597-2150-4d53-a529-9ef782740892 None admin] PATH_INFO: `/v3/domains/fcbd782c7f7b49158dfc3fecbb2a9596` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:46.871951 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5ee597-2150-4d53-a529-9ef782740892 None admin] RBAC: Authorizing `identity:get_domain(domain_id=fcbd782c7f7b49158dfc3fecbb2a9596)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:46.873378 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5ee597-2150-4d53-a529-9ef782740892 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:46.875326 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 596/1176] 10.222.0.71 () {56 vars in 1183 bytes} [Fri Aug 26 08:39:46 2022] GET /identity/v3/domains/fcbd782c7f7b49158dfc3fecbb2a9596 => generated 267 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:46.882194 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017641bb-33ea-4d6e-b46b-a86eb5c396f8 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:46.917473 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:47.016988 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017641bb-33ea-4d6e-b46b-a86eb5c396f8 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:47.066101 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6df00-71c7-4714-9cbe-505323f36069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:47.066705 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6df00-71c7-4714-9cbe-505323f36069 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:47.067009 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6df00-71c7-4714-9cbe-505323f36069 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:47.067301 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6df00-71c7-4714-9cbe-505323f36069 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:47.067953 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6df00-71c7-4714-9cbe-505323f36069 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:47.069421 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6df00-71c7-4714-9cbe-505323f36069 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:47.094231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 581/1177] 10.222.0.71 () {60 vars in 1132 bytes} [Fri Aug 26 08:39:46 2022] POST /identity/v3/users => generated 311 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:39:49.039085 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89617541-bba0-4e89-b132-9e5060f8847c None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:49.039661 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89617541-bba0-4e89-b132-9e5060f8847c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:49.039661 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89617541-bba0-4e89-b132-9e5060f8847c None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:49.041163 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 597/1178] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:39:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:39:49.047768 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628576e-f3ed-4117-8ec1-7c05165ccb97 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:49.048527 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628576e-f3ed-4117-8ec1-7c05165ccb97 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:49.048679 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628576e-f3ed-4117-8ec1-7c05165ccb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:49.072332 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-d628576e-f3ed-4117-8ec1-7c05165ccb97 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:39:49.073556 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d628576e-f3ed-4117-8ec1-7c05165ccb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:39:49.151298 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 582/1179] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:39:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:39:49.425077 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89617541-bba0-4e89-b132-9e5060f8847c None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:49.542549 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89617541-bba0-4e89-b132-9e5060f8847c None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:49.581303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9dac27-b961-4fc2-bf6a-a5da775cf458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:49.581949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9dac27-b961-4fc2-bf6a-a5da775cf458 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:49.582223 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9dac27-b961-4fc2-bf6a-a5da775cf458 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:49.582487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9dac27-b961-4fc2-bf6a-a5da775cf458 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:49.593198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9dac27-b961-4fc2-bf6a-a5da775cf458 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:39:49.597908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-dd9dac27-b961-4fc2-bf6a-a5da775cf458 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:39:49.597908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:39:49.597908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:39:49.597908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:39:49.597908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:39:49.597908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:39:49.597908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:39:49.597908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:39:49.597908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:39:49.597908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:39:49.597908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:39:49.597908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:39:49.597908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:39:49.597908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:39:49.597908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:39:49.597908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:39:49.599071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:39:49.599071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:39:49.599071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:39:49.599071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:39:49.599071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:39:49.599071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:39:49.599071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:39:49.599071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:39:49.599071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:39:49.599071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:39:49.599071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:39:49.599071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:39:49.599071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:39:49.599071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:39:49.599071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:39:49.599071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:39:49.600145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:39:49.600145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:39:49.600145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:39:49.600145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:39:49.600145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:39:49.600145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:39:49.600145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:39:49.600145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:39:49.600145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:39:49.600145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:39:49.600145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:39:49.600145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:39:49.600145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 598/1180] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:39:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:39:49.605643 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628576e-f3ed-4117-8ec1-7c05165ccb97 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:49.621418 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:49.708371 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628576e-f3ed-4117-8ec1-7c05165ccb97 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:49.745404 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e02df-3a61-47e8-8589-3b88ee91f2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:49.746161 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e02df-3a61-47e8-8589-3b88ee91f2d7 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:49.746530 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e02df-3a61-47e8-8589-3b88ee91f2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:49.746945 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e02df-3a61-47e8-8589-3b88ee91f2d7 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:49.747937 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e02df-3a61-47e8-8589-3b88ee91f2d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:49.748294 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e02df-3a61-47e8-8589-3b88ee91f2d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:49.750827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e02df-3a61-47e8-8589-3b88ee91f2d7 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:49.763490 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 583/1181] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:39:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:49.769750 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9dac27-b961-4fc2-bf6a-a5da775cf458 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:49.774290 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:49.876937 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9dac27-b961-4fc2-bf6a-a5da775cf458 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:49.912741 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3634b-1c03-4768-92fd-9b6ae5541963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:49.913378 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3634b-1c03-4768-92fd-9b6ae5541963 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:49.913696 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3634b-1c03-4768-92fd-9b6ae5541963 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:49.913979 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3634b-1c03-4768-92fd-9b6ae5541963 None admin] PATH_INFO: `/v3/users/28d66faaf6ca4b2d8025f04fa9c6f709` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:49.928799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c3634b-1c03-4768-92fd-9b6ae5541963 None admin] RBAC: Authorizing `identity:get_user(user_id=28d66faaf6ca4b2d8025f04fa9c6f709)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:49.930409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c3634b-1c03-4768-92fd-9b6ae5541963 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:49.932279 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 599/1182] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:39:49 2022] GET /identity/v3/users/28d66faaf6ca4b2d8025f04fa9c6f709 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:49.939174 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e02df-3a61-47e8-8589-3b88ee91f2d7 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:50.025544 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:50.051638 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e02df-3a61-47e8-8589-3b88ee91f2d7 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:50.088879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22403b93-e809-4f47-98ba-582a34463e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:50.089367 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22403b93-e809-4f47-98ba-582a34463e1c None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:50.089663 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22403b93-e809-4f47-98ba-582a34463e1c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:50.089930 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22403b93-e809-4f47-98ba-582a34463e1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:50.157915 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22403b93-e809-4f47-98ba-582a34463e1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:50.160732 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22403b93-e809-4f47-98ba-582a34463e1c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:50.238596 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:50.266241 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 584/1183] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:39:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 08:39:50.272570 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3634b-1c03-4768-92fd-9b6ae5541963 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:50.336416 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:50.389312 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3634b-1c03-4768-92fd-9b6ae5541963 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:50.432520 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad0efe-f46d-4fc1-9dcd-69a45ff744f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:50.433455 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad0efe-f46d-4fc1-9dcd-69a45ff744f3 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:50.433883 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad0efe-f46d-4fc1-9dcd-69a45ff744f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:50.434263 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad0efe-f46d-4fc1-9dcd-69a45ff744f3 None admin] PATH_INFO: `/v3/projects/937c7950e75c4a78a790c9fc742f4a17` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:50.436102 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad0efe-f46d-4fc1-9dcd-69a45ff744f3 None admin] RBAC: Authorizing `identity:get_project(project_id=937c7950e75c4a78a790c9fc742f4a17)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:50.438339 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad0efe-f46d-4fc1-9dcd-69a45ff744f3 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:50.440947 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 600/1184] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:39:50 2022] GET /identity/v3/projects/937c7950e75c4a78a790c9fc742f4a17 => generated 361 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:50.449631 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22403b93-e809-4f47-98ba-582a34463e1c None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:50.558393 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22403b93-e809-4f47-98ba-582a34463e1c None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:50.611806 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dacdb4-e11f-404a-995b-fa9a3338ed32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:50.612480 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dacdb4-e11f-404a-995b-fa9a3338ed32 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:50.612854 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dacdb4-e11f-404a-995b-fa9a3338ed32 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:50.613121 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dacdb4-e11f-404a-995b-fa9a3338ed32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:50.614232 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dacdb4-e11f-404a-995b-fa9a3338ed32 None admin] RBAC: Adding query filter params (role.id=18c3c8a821b647f7982a955880bd0a9a, scope.project.id=937c7950e75c4a78a790c9fc742f4a17, user.id=28d66faaf6ca4b2d8025f04fa9c6f709) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:50.614611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dacdb4-e11f-404a-995b-fa9a3338ed32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:50.616697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dacdb4-e11f-404a-995b-fa9a3338ed32 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:50.622994 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 585/1185] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:39:50 2022] GET /identity/v3/role_assignments?user.id=28d66faaf6ca4b2d8025f04fa9c6f709&scope.project.id=937c7950e75c4a78a790c9fc742f4a17&role.id=18c3c8a821b647f7982a955880bd0a9a => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:52.510224 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9bf11-d923-49aa-bcd9-60e139431e68 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:52.510858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9bf11-d923-49aa-bcd9-60e139431e68 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:52.510858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9bf11-d923-49aa-bcd9-60e139431e68 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:52.512149 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 601/1186] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:39:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:39:52.517805 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1edc77-d9d0-4884-a7c1-672bb04163a3 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:52.518275 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1edc77-d9d0-4884-a7c1-672bb04163a3 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:52.518373 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1edc77-d9d0-4884-a7c1-672bb04163a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:52.538502 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-1b1edc77-d9d0-4884-a7c1-672bb04163a3 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:39:52.539749 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1b1edc77-d9d0-4884-a7c1-672bb04163a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:39:52.571476 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:52.625153 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 586/1187] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:39:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 08:39:52.893897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9bf11-d923-49aa-bcd9-60e139431e68 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:53.002909 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9bf11-d923-49aa-bcd9-60e139431e68 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:53.031227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:53.037498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3638f-6a98-4e99-8358-f4b278eb507b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:53.038124 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3638f-6a98-4e99-8358-f4b278eb507b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:53.038398 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3638f-6a98-4e99-8358-f4b278eb507b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:53.038663 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3638f-6a98-4e99-8358-f4b278eb507b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:53.050407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3638f-6a98-4e99-8358-f4b278eb507b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:39:53.055112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-d2a3638f-6a98-4e99-8358-f4b278eb507b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:39:53.055112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:39:53.055112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:39:53.055112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:39:53.055112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:39:53.055112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:39:53.055112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:39:53.055112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:39:53.055112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:39:53.055112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:39:53.055112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:39:53.055112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:39:53.055112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:39:53.055112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:39:53.055112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:39:53.055112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:39:53.056595 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:39:53.056595 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:39:53.056595 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:39:53.056595 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:39:53.056595 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:39:53.056595 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:39:53.056595 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:39:53.056595 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:39:53.056595 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:39:53.056595 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:39:53.056595 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:39:53.056595 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:39:53.056595 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:39:53.056595 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:39:53.056595 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:39:53.056595 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:39:53.057689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:39:53.057689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:39:53.057689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:39:53.057689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:39:53.057689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:39:53.057689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:39:53.057689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:39:53.057689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:39:53.057689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:39:53.057689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:39:53.057689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:39:53.057689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:39:53.059063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 602/1188] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:39:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:39:53.062553 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1edc77-d9d0-4884-a7c1-672bb04163a3 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:53.176997 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1edc77-d9d0-4884-a7c1-672bb04163a3 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:53.215525 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1671fea-e038-468f-9495-8c69f7d78bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:53.216121 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1671fea-e038-468f-9495-8c69f7d78bb0 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:53.216391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1671fea-e038-468f-9495-8c69f7d78bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:53.216654 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1671fea-e038-468f-9495-8c69f7d78bb0 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:53.217316 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1671fea-e038-468f-9495-8c69f7d78bb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:53.217619 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1671fea-e038-468f-9495-8c69f7d78bb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:53.219011 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1671fea-e038-468f-9495-8c69f7d78bb0 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:53.228496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 587/1189] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:39:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:53.234805 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3638f-6a98-4e99-8358-f4b278eb507b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:53.357523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3638f-6a98-4e99-8358-f4b278eb507b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:53.400844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1112e9-659e-4181-8ee9-1feea95ca540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:53.401474 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1112e9-659e-4181-8ee9-1feea95ca540 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:53.401753 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1112e9-659e-4181-8ee9-1feea95ca540 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:53.402029 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1112e9-659e-4181-8ee9-1feea95ca540 None admin] PATH_INFO: `/v3/users/28d66faaf6ca4b2d8025f04fa9c6f709` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:53.403202 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1112e9-659e-4181-8ee9-1feea95ca540 None admin] RBAC: Authorizing `identity:get_user(user_id=28d66faaf6ca4b2d8025f04fa9c6f709)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:53.404634 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1112e9-659e-4181-8ee9-1feea95ca540 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:53.406617 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 603/1190] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:39:53 2022] GET /identity/v3/users/28d66faaf6ca4b2d8025f04fa9c6f709 => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:53.415465 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1671fea-e038-468f-9495-8c69f7d78bb0 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:53.449133 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:53.523072 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1671fea-e038-468f-9495-8c69f7d78bb0 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:53.559489 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496cd893-4b37-480a-9e73-f03fde9ea323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:53.560127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cd893-4b37-480a-9e73-f03fde9ea323 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:53.560406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cd893-4b37-480a-9e73-f03fde9ea323 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:53.560678 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cd893-4b37-480a-9e73-f03fde9ea323 None admin] PATH_INFO: `/v3/projects/937c7950e75c4a78a790c9fc742f4a17` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:53.561846 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496cd893-4b37-480a-9e73-f03fde9ea323 None admin] RBAC: Authorizing `identity:get_project(project_id=937c7950e75c4a78a790c9fc742f4a17)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:53.563319 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496cd893-4b37-480a-9e73-f03fde9ea323 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:53.564968 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 588/1191] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:39:53 2022] GET /identity/v3/projects/937c7950e75c4a78a790c9fc742f4a17 => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:53.574360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1112e9-659e-4181-8ee9-1feea95ca540 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:53.642298 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:53.685594 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1112e9-659e-4181-8ee9-1feea95ca540 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:53.722251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b683b4-a992-4e3f-a4fd-66aa2771ffbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:53.722824 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b683b4-a992-4e3f-a4fd-66aa2771ffbc None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:53.723130 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b683b4-a992-4e3f-a4fd-66aa2771ffbc None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:53.723387 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b683b4-a992-4e3f-a4fd-66aa2771ffbc None admin] PATH_INFO: `/v3/projects/937c7950e75c4a78a790c9fc742f4a17/users/28d66faaf6ca4b2d8025f04fa9c6f709/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:53.725362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b683b4-a992-4e3f-a4fd-66aa2771ffbc None admin] RBAC: Authorizing `identity:create_grant(project_id=937c7950e75c4a78a790c9fc742f4a17, user_id=28d66faaf6ca4b2d8025f04fa9c6f709, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:53.726843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b683b4-a992-4e3f-a4fd-66aa2771ffbc None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:53.744805 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 604/1192] 10.222.0.71 () {58 vars in 1439 bytes} [Fri Aug 26 08:39:53 2022] PUT /identity/v3/projects/937c7950e75c4a78a790c9fc742f4a17/users/28d66faaf6ca4b2d8025f04fa9c6f709/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:39:55.625951 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec69df3-e2b2-4821-853c-84cf000a0a57 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:55.626750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec69df3-e2b2-4821-853c-84cf000a0a57 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:55.626750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec69df3-e2b2-4821-853c-84cf000a0a57 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:55.627784 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 589/1193] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:39:55.634684 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e4d23-6d26-4b57-9bc9-9bca1d61e388 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:55.635479 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e4d23-6d26-4b57-9bc9-9bca1d61e388 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:55.635633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e4d23-6d26-4b57-9bc9-9bca1d61e388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:55.667177 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-404e4d23-6d26-4b57-9bc9-9bca1d61e388 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:39:55.668381 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-404e4d23-6d26-4b57-9bc9-9bca1d61e388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:39:55.737971 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 605/1194] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:39:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:39:56.012366 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec69df3-e2b2-4821-853c-84cf000a0a57 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:56.125351 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:56.130408 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec69df3-e2b2-4821-853c-84cf000a0a57 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:56.166905 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a3423e-b664-4dbb-980c-b4c4ae6c1d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:56.167542 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3423e-b664-4dbb-980c-b4c4ae6c1d26 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:56.167822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3423e-b664-4dbb-980c-b4c4ae6c1d26 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:56.168096 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3423e-b664-4dbb-980c-b4c4ae6c1d26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:56.183822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a3423e-b664-4dbb-980c-b4c4ae6c1d26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:39:56.191566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-42a3423e-b664-4dbb-980c-b4c4ae6c1d26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:39:56.191566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:39:56.191566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:39:56.191566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:39:56.191566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:39:56.191566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:39:56.191566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:39:56.191566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:39:56.191566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:39:56.191566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:39:56.191566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:39:56.191566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:39:56.191566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:39:56.191566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:39:56.191566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:39:56.191566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:39:56.192677 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:39:56.192677 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:39:56.192677 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:39:56.192677 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:39:56.192677 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:39:56.192677 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:39:56.192677 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:39:56.192677 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:39:56.192677 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:39:56.192677 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:39:56.192677 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:39:56.192677 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:39:56.192677 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:39:56.192677 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:39:56.192677 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:39:56.192677 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:39:56.193636 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:39:56.193636 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:39:56.193636 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:39:56.193636 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:39:56.193636 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:39:56.193636 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:39:56.193636 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:39:56.193636 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:39:56.193636 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:39:56.193636 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:39:56.193636 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:39:56.193636 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:39:56.193636 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 590/1195] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:39:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:39:56.199470 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e4d23-6d26-4b57-9bc9-9bca1d61e388 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:56.263364 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:56.329199 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e4d23-6d26-4b57-9bc9-9bca1d61e388 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:56.371283 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a7b49e-944a-48ba-a730-70d01631060b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:56.371933 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7b49e-944a-48ba-a730-70d01631060b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:56.372247 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7b49e-944a-48ba-a730-70d01631060b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:56.372550 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7b49e-944a-48ba-a730-70d01631060b None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:56.373261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a7b49e-944a-48ba-a730-70d01631060b None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:56.373597 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a7b49e-944a-48ba-a730-70d01631060b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:56.375543 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a7b49e-944a-48ba-a730-70d01631060b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:56.384915 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 606/1196] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:39:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:56.391300 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a3423e-b664-4dbb-980c-b4c4ae6c1d26 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:56.507763 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a3423e-b664-4dbb-980c-b4c4ae6c1d26 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:56.542796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70269b4e-6561-476f-9e11-a55bd44be427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:56.543369 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70269b4e-6561-476f-9e11-a55bd44be427 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:56.543644 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70269b4e-6561-476f-9e11-a55bd44be427 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:56.543909 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70269b4e-6561-476f-9e11-a55bd44be427 None admin] PATH_INFO: `/v3/users/28d66faaf6ca4b2d8025f04fa9c6f709` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:56.545144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70269b4e-6561-476f-9e11-a55bd44be427 None admin] RBAC: Authorizing `identity:get_user(user_id=28d66faaf6ca4b2d8025f04fa9c6f709)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:56.546577 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70269b4e-6561-476f-9e11-a55bd44be427 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:56.548222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 591/1197] 10.222.0.71 () {56 vars in 1177 bytes} [Fri Aug 26 08:39:56 2022] GET /identity/v3/users/28d66faaf6ca4b2d8025f04fa9c6f709 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:56.557219 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a7b49e-944a-48ba-a730-70d01631060b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:56.664312 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a7b49e-944a-48ba-a730-70d01631060b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:56.674926 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:56.699754 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c08616-d7d0-4491-99a4-70f731d8db2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:56.700365 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c08616-d7d0-4491-99a4-70f731d8db2b None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:56.700638 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c08616-d7d0-4491-99a4-70f731d8db2b None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:56.700893 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c08616-d7d0-4491-99a4-70f731d8db2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:56.777192 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c08616-d7d0-4491-99a4-70f731d8db2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:56.778854 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c08616-d7d0-4491-99a4-70f731d8db2b None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:56.892559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 607/1198] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:39:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 08:39:56.899550 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70269b4e-6561-476f-9e11-a55bd44be427 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:56.937095 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:57.018793 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70269b4e-6561-476f-9e11-a55bd44be427 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:57.054513 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f28409-d869-41b2-8a0a-f170418a0b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:57.055174 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f28409-d869-41b2-8a0a-f170418a0b51 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:57.055445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f28409-d869-41b2-8a0a-f170418a0b51 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:57.055717 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f28409-d869-41b2-8a0a-f170418a0b51 None admin] PATH_INFO: `/v3/projects/937c7950e75c4a78a790c9fc742f4a17` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:57.056931 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f28409-d869-41b2-8a0a-f170418a0b51 None admin] RBAC: Authorizing `identity:get_project(project_id=937c7950e75c4a78a790c9fc742f4a17)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:57.058369 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f28409-d869-41b2-8a0a-f170418a0b51 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:57.060099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 592/1199] 10.222.0.71 () {56 vars in 1186 bytes} [Fri Aug 26 08:39:56 2022] GET /identity/v3/projects/937c7950e75c4a78a790c9fc742f4a17 => generated 361 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:39:57.070634 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c08616-d7d0-4491-99a4-70f731d8db2b None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:39:57.084357 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:39:57.179914 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c08616-d7d0-4491-99a4-70f731d8db2b None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:39:57.216968 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121f502-5e1e-4274-8ee4-e07d56b53eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:39:57.217583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121f502-5e1e-4274-8ee4-e07d56b53eb1 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:39:57.217862 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121f502-5e1e-4274-8ee4-e07d56b53eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:39:57.218128 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121f502-5e1e-4274-8ee4-e07d56b53eb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:39:57.218869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121f502-5e1e-4274-8ee4-e07d56b53eb1 None admin] RBAC: Adding query filter params (role.id=18c3c8a821b647f7982a955880bd0a9a, scope.project.id=937c7950e75c4a78a790c9fc742f4a17, user.id=28d66faaf6ca4b2d8025f04fa9c6f709) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:39:57.219211 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121f502-5e1e-4274-8ee4-e07d56b53eb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:39:57.220643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121f502-5e1e-4274-8ee4-e07d56b53eb1 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:39:57.227198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 608/1200] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:39:57 2022] GET /identity/v3/role_assignments?user.id=28d66faaf6ca4b2d8025f04fa9c6f709&scope.project.id=937c7950e75c4a78a790c9fc742f4a17&role.id=18c3c8a821b647f7982a955880bd0a9a => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:40:07.049702 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=53 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:40:07.223953 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=63 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:40:54.589482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a14a4-f4ed-43ee-b05a-917fac56ceaa None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:40:54.589482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a14a4-f4ed-43ee-b05a-917fac56ceaa None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:40:54.589482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a14a4-f4ed-43ee-b05a-917fac56ceaa None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:40:54.590576 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 593/1201] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:40:54.596611 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e49399-5573-43d6-a23a-8968455a359b None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:40:54.597119 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e49399-5573-43d6-a23a-8968455a359b None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:40:54.597206 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e49399-5573-43d6-a23a-8968455a359b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:40:54.617662 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e4e49399-5573-43d6-a23a-8968455a359b None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:40:54.619113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e4e49399-5573-43d6-a23a-8968455a359b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:40:54.720656 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 609/1202] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:40:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:40:54.994150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a14a4-f4ed-43ee-b05a-917fac56ceaa None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:40:55.154047 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a14a4-f4ed-43ee-b05a-917fac56ceaa None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:40:55.206896 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917775a5-52db-4120-865f-5f019e0aae8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:40:55.207726 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917775a5-52db-4120-865f-5f019e0aae8e None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:40:55.207996 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917775a5-52db-4120-865f-5f019e0aae8e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:40:55.208269 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917775a5-52db-4120-865f-5f019e0aae8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:40:55.225008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917775a5-52db-4120-865f-5f019e0aae8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:40:55.226498 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917775a5-52db-4120-865f-5f019e0aae8e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:40:55.245451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-917775a5-52db-4120-865f-5f019e0aae8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:40:55.245451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:40:55.245451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:40:55.245451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:40:55.245451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:40:55.245451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:40:55.245451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:40:55.245451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:40:55.245451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:40:55.245451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:40:55.245451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:40:55.245451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:40:55.245451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:40:55.245451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:40:55.245451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:40:55.245451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:40:55.252113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:40:55.252113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:40:55.252113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:40:55.252113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:40:55.252113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:40:55.252113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:40:55.252113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:40:55.252113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:40:55.252113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:40:55.252113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:40:55.252113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:40:55.252113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:40:55.252113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:40:55.252113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:40:55.252113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:40:55.252113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:40:55.252113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:40:55.253761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:40:55.253761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:40:55.253761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:40:55.253761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 594/1203] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:40:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:40:55.256420 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e49399-5573-43d6-a23a-8968455a359b None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:40:55.439882 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e49399-5573-43d6-a23a-8968455a359b None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:40:55.480246 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:40:55.491778 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65606c90-0c02-47f6-8dde-7bf5ae03713a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:40:55.493039 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65606c90-0c02-47f6-8dde-7bf5ae03713a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:40:55.493541 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65606c90-0c02-47f6-8dde-7bf5ae03713a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:40:55.494034 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65606c90-0c02-47f6-8dde-7bf5ae03713a None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:40:55.495331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65606c90-0c02-47f6-8dde-7bf5ae03713a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:40:55.495797 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65606c90-0c02-47f6-8dde-7bf5ae03713a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:40:55.498129 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65606c90-0c02-47f6-8dde-7bf5ae03713a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:40:55.522926 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 610/1204] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:40:55.530169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917775a5-52db-4120-865f-5f019e0aae8e None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:40:55.617937 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:40:55.710046 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917775a5-52db-4120-865f-5f019e0aae8e None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:40:55.774762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795818a6-881d-402e-adfd-743ebb11149c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:40:55.775718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795818a6-881d-402e-adfd-743ebb11149c None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:40:55.776247 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795818a6-881d-402e-adfd-743ebb11149c None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:40:55.777107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795818a6-881d-402e-adfd-743ebb11149c None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:40:55.778210 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795818a6-881d-402e-adfd-743ebb11149c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:40:55.781695 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795818a6-881d-402e-adfd-743ebb11149c None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:40:55.834919 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 595/1205] 10.222.0.71 () {60 vars in 1132 bytes} [Fri Aug 26 08:40:55 2022] POST /identity/v3/users => generated 251 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:40:58.061020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db98098-2fe9-4bdf-ae96-79f8968457a0 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:40:58.061609 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db98098-2fe9-4bdf-ae96-79f8968457a0 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:40:58.061609 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db98098-2fe9-4bdf-ae96-79f8968457a0 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:40:58.063187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 611/1206] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:40:58.070352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335eaf08-0109-4641-965c-700a1c51db34 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:40:58.071129 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335eaf08-0109-4641-965c-700a1c51db34 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:40:58.071129 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335eaf08-0109-4641-965c-700a1c51db34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:40:58.095761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-335eaf08-0109-4641-965c-700a1c51db34 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:40:58.097213 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-335eaf08-0109-4641-965c-700a1c51db34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:40:58.214983 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 596/1207] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:40:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:40:58.506880 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db98098-2fe9-4bdf-ae96-79f8968457a0 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:40:58.674003 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db98098-2fe9-4bdf-ae96-79f8968457a0 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:40:58.718071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21774e-050d-4bfb-902c-9f93d0148069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:40:58.718846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21774e-050d-4bfb-902c-9f93d0148069 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:40:58.719149 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21774e-050d-4bfb-902c-9f93d0148069 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:40:58.719423 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21774e-050d-4bfb-902c-9f93d0148069 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:40:58.732614 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd21774e-050d-4bfb-902c-9f93d0148069 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:40:58.738032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-cd21774e-050d-4bfb-902c-9f93d0148069 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:40:58.738032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:40:58.738032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:40:58.738032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:40:58.738032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:40:58.738032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:40:58.738032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:40:58.738032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:40:58.738032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:40:58.738032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:40:58.738032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:40:58.738032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:40:58.738032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:40:58.738032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:40:58.738032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:40:58.738032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:40:58.739314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:40:58.739314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:40:58.739314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:40:58.739314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:40:58.739314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:40:58.739314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:40:58.739314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:40:58.739314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:40:58.739314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:40:58.739314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:40:58.739314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:40:58.739314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:40:58.739314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:40:58.739314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:40:58.739314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:40:58.739314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:40:58.740813 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:40:58.740813 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:40:58.740813 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:40:58.740813 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:40:58.740813 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:40:58.740813 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:40:58.740813 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:40:58.740813 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:40:58.740813 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:40:58.740813 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:40:58.740813 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:40:58.740813 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:40:58.740813 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 612/1208] 10.222.0.71 () {56 vars in 1102 bytes} [Fri Aug 26 08:40:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:40:58.745971 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335eaf08-0109-4641-965c-700a1c51db34 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:40:58.814323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:40:58.897351 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335eaf08-0109-4641-965c-700a1c51db34 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:40:58.958471 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e78fe-3cc6-4fa5-9795-171b6b47def8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:40:58.958471 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e78fe-3cc6-4fa5-9795-171b6b47def8 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:40:58.958471 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e78fe-3cc6-4fa5-9795-171b6b47def8 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:40:58.958848 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e78fe-3cc6-4fa5-9795-171b6b47def8 None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:40:58.959759 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149e78fe-3cc6-4fa5-9795-171b6b47def8 None admin] RBAC: Adding query filter params (name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:40:58.960260 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149e78fe-3cc6-4fa5-9795-171b6b47def8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:40:58.962570 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149e78fe-3cc6-4fa5-9795-171b6b47def8 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:40:58.973262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 597/1209] 10.222.0.71 () {56 vars in 1103 bytes} [Fri Aug 26 08:40:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:40:58.979750 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21774e-050d-4bfb-902c-9f93d0148069 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:40:59.050430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:40:59.129619 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21774e-050d-4bfb-902c-9f93d0148069 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:40:59.179386 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d973082-3705-4964-825f-cc2bd9e9cb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:40:59.180050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d973082-3705-4964-825f-cc2bd9e9cb90 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:40:59.180328 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d973082-3705-4964-825f-cc2bd9e9cb90 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:40:59.180587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d973082-3705-4964-825f-cc2bd9e9cb90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:40:59.199220 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d973082-3705-4964-825f-cc2bd9e9cb90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:40:59.200897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d973082-3705-4964-825f-cc2bd9e9cb90 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:40:59.221633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-6d973082-3705-4964-825f-cc2bd9e9cb90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:40:59.221633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:40:59.221633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:40:59.221633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:40:59.221633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:40:59.221633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:40:59.221633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:40:59.221633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:40:59.221633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:40:59.221633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:40:59.221633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:40:59.221633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:40:59.221633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:40:59.221633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:40:59.221633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:40:59.221633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:40:59.222957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:40:59.222957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:40:59.222957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:40:59.222957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:40:59.222957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:40:59.222957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:40:59.222957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:40:59.222957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:40:59.222957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:40:59.222957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:40:59.222957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:40:59.222957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:40:59.222957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:40:59.222957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:40:59.222957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:40:59.222957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:40:59.222957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:40:59.224347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:40:59.224347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:40:59.224347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:40:59.224347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 613/1210] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:40:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:40:59.228952 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e78fe-3cc6-4fa5-9795-171b6b47def8 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:40:59.374696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:40:59.378372 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e78fe-3cc6-4fa5-9795-171b6b47def8 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:40:59.427765 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88847c1-c063-44e5-b6af-c1855d955b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:40:59.428638 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88847c1-c063-44e5-b6af-c1855d955b1f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:40:59.428760 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88847c1-c063-44e5-b6af-c1855d955b1f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:40:59.429057 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88847c1-c063-44e5-b6af-c1855d955b1f None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:40:59.429801 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88847c1-c063-44e5-b6af-c1855d955b1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:40:59.430185 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88847c1-c063-44e5-b6af-c1855d955b1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:40:59.431720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88847c1-c063-44e5-b6af-c1855d955b1f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:40:59.447486 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 598/1211] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:40:59.455456 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d973082-3705-4964-825f-cc2bd9e9cb90 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:40:59.601415 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d973082-3705-4964-825f-cc2bd9e9cb90 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:40:59.603124 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:40:59.643398 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72bbc8e-ec5f-466d-885a-42897aaa6510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:40:59.644659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72bbc8e-ec5f-466d-885a-42897aaa6510 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:40:59.644659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72bbc8e-ec5f-466d-885a-42897aaa6510 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:40:59.644843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72bbc8e-ec5f-466d-885a-42897aaa6510 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:40:59.654041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72bbc8e-ec5f-466d-885a-42897aaa6510 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:40:59.655626 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72bbc8e-ec5f-466d-885a-42897aaa6510 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:40:59.670147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-e72bbc8e-ec5f-466d-885a-42897aaa6510 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 08:40:59.670147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:40:59.670147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:40:59.670147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:40:59.670147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:40:59.670147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:40:59.670147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:40:59.670147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:40:59.670147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:40:59.670147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:40:59.670147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:40:59.670147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:40:59.670147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:40:59.670147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:40:59.670147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:40:59.670147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:40:59.671334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:40:59.671334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:40:59.671334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:40:59.671334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:40:59.671334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:40:59.671334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:40:59.671334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:40:59.671334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:40:59.671334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:40:59.671334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:40:59.671334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:40:59.671334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:40:59.671334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:40:59.671334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:40:59.671334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:40:59.671334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:40:59.671334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:40:59.671334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:40:59.672413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:40:59.672413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:40:59.672413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:40:59.672413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:40:59.672413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:40:59.672413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:40:59.672413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:40:59.672413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:40:59.672413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 08:40:59.672413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:40:59.672413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 614/1212] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:40:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:40:59.677851 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88847c1-c063-44e5-b6af-c1855d955b1f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:40:59.801876 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88847c1-c063-44e5-b6af-c1855d955b1f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:40:59.843083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a3efc-0dc3-4c66-be4c-2b2a4d18bf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:40:59.843869 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a3efc-0dc3-4c66-be4c-2b2a4d18bf59 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:40:59.844140 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a3efc-0dc3-4c66-be4c-2b2a4d18bf59 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:40:59.844294 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a3efc-0dc3-4c66-be4c-2b2a4d18bf59 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:40:59.845035 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a3efc-0dc3-4c66-be4c-2b2a4d18bf59 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:40:59.845334 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a3efc-0dc3-4c66-be4c-2b2a4d18bf59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:40:59.846769 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a3efc-0dc3-4c66-be4c-2b2a4d18bf59 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:40:59.870083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 599/1213] 10.222.0.71 () {56 vars in 1143 bytes} [Fri Aug 26 08:40:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:40:59.871530 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72bbc8e-ec5f-466d-885a-42897aaa6510 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:00.011531 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72bbc8e-ec5f-466d-885a-42897aaa6510 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:00.070042 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc48356-f232-4981-bc48-8378c1b10863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:00.071085 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc48356-f232-4981-bc48-8378c1b10863 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:00.071552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc48356-f232-4981-bc48-8378c1b10863 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:00.071995 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc48356-f232-4981-bc48-8378c1b10863 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:00.115034 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:00.163626 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc48356-f232-4981-bc48-8378c1b10863 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:00.165393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc48356-f232-4981-bc48-8378c1b10863 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:00.294914 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 615/1214] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:40:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 08:41:00.297854 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a3efc-0dc3-4c66-be4c-2b2a4d18bf59 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:00.320849 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:00.426830 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a3efc-0dc3-4c66-be4c-2b2a4d18bf59 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:00.469415 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627da72e-1f3f-4275-9289-1becdc896a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:00.470559 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627da72e-1f3f-4275-9289-1becdc896a72 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:00.470559 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627da72e-1f3f-4275-9289-1becdc896a72 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:00.470772 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627da72e-1f3f-4275-9289-1becdc896a72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:00.484238 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627da72e-1f3f-4275-9289-1becdc896a72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:00.485801 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627da72e-1f3f-4275-9289-1becdc896a72 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:00.503470 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-627da72e-1f3f-4275-9289-1becdc896a72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:41:00.503470 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:00.503470 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:00.503470 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:00.503470 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:00.503470 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:00.503470 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:00.503470 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:00.503470 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:00.503470 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:00.503470 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:00.503470 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:00.503470 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:41:00.503470 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:41:00.503470 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:41:00.503470 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:41:00.504764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:00.504764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:00.504764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:00.504764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:00.504764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:00.504764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:00.504764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:00.504764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:00.504764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:00.504764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:00.504764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:00.504764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:00.504764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:00.504764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:00.504764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:00.504764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:00.504764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:41:00.506332 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:41:00.506332 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:41:00.506332 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:41:00.506332 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 600/1215] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:41:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:41:00.511535 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc48356-f232-4981-bc48-8378c1b10863 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:00.593813 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:00.644182 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc48356-f232-4981-bc48-8378c1b10863 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:00.689635 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156bcb8-22a4-4705-8078-8e5e529540ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:00.690184 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156bcb8-22a4-4705-8078-8e5e529540ab None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:00.690539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156bcb8-22a4-4705-8078-8e5e529540ab None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:00.690814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156bcb8-22a4-4705-8078-8e5e529540ab None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:00.691512 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156bcb8-22a4-4705-8078-8e5e529540ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:00.691817 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156bcb8-22a4-4705-8078-8e5e529540ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:00.696319 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156bcb8-22a4-4705-8078-8e5e529540ab None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:00.711472 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 616/1216] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:41:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:00.717982 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627da72e-1f3f-4275-9289-1becdc896a72 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:00.820973 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:00.872818 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627da72e-1f3f-4275-9289-1becdc896a72 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:00.923981 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350ed78-1223-43d9-9e85-5e13ae0d9cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:00.924634 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350ed78-1223-43d9-9e85-5e13ae0d9cf1 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:00.924900 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350ed78-1223-43d9-9e85-5e13ae0d9cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:00.925184 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350ed78-1223-43d9-9e85-5e13ae0d9cf1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:00.932805 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b350ed78-1223-43d9-9e85-5e13ae0d9cf1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:00.934373 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b350ed78-1223-43d9-9e85-5e13ae0d9cf1 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:00.947287 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-b350ed78-1223-43d9-9e85-5e13ae0d9cf1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:41:00.947287 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:00.947287 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:00.947287 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:00.947287 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:00.947287 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:00.947287 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:00.947287 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:00.947287 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:00.947287 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:00.947287 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:00.947287 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:00.947287 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:41:00.947287 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:41:00.947287 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:41:00.948262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:41:00.948262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:00.948262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:00.948262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:00.948262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:00.948262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:00.948262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:00.948262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:00.948262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:00.948262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:00.948262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:00.948262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:00.948262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:00.948262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:00.948262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:00.948262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:00.948262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:00.950130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:41:00.950130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:41:00.950130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:41:00.950130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:41:00.950130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:41:00.950130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:41:00.950130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:41:00.950130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:41:00.950130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 601/1217] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:41:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:41:00.956133 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156bcb8-22a4-4705-8078-8e5e529540ab None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:01.068109 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156bcb8-22a4-4705-8078-8e5e529540ab None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:01.093340 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:01.106901 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251e6de4-24f9-4ce1-a4b2-e1a0da74ddca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:01.107536 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e6de4-24f9-4ce1-a4b2-e1a0da74ddca None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:01.107802 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e6de4-24f9-4ce1-a4b2-e1a0da74ddca None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:01.108094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e6de4-24f9-4ce1-a4b2-e1a0da74ddca None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:01.108798 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e6de4-24f9-4ce1-a4b2-e1a0da74ddca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:01.109104 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e6de4-24f9-4ce1-a4b2-e1a0da74ddca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:01.110719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e6de4-24f9-4ce1-a4b2-e1a0da74ddca None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:01.124861 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 617/1218] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:41:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:01.131597 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350ed78-1223-43d9-9e85-5e13ae0d9cf1 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:01.241835 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350ed78-1223-43d9-9e85-5e13ae0d9cf1 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:01.279913 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:01.287107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43798e72-ea7f-4723-8da2-d430982d43ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:01.287843 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43798e72-ea7f-4723-8da2-d430982d43ab None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:01.288191 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43798e72-ea7f-4723-8da2-d430982d43ab None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:01.288523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43798e72-ea7f-4723-8da2-d430982d43ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:01.289368 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43798e72-ea7f-4723-8da2-d430982d43ab None admin] RBAC: Adding query filter params (role.id=33ac296b51c143e7bb44211175752edb, scope.project.id=543d44293d894a7fbb83401f7a26dd9b, user.id=0cf5e2b4f5e440a7942adea3d51cec75) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:01.289772 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43798e72-ea7f-4723-8da2-d430982d43ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:01.291575 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43798e72-ea7f-4723-8da2-d430982d43ab None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:01.310919 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 602/1219] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:41:01 2022] GET /identity/v3/role_assignments?user.id=0cf5e2b4f5e440a7942adea3d51cec75&scope.project.id=543d44293d894a7fbb83401f7a26dd9b&role.id=33ac296b51c143e7bb44211175752edb => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:03.339855 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42a264-ee04-41ee-9d4d-8f7f614fa72a None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:03.340442 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42a264-ee04-41ee-9d4d-8f7f614fa72a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:03.340620 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42a264-ee04-41ee-9d4d-8f7f614fa72a None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:03.341866 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 618/1220] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:41:03.348375 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74626f1-498a-43c8-9b5c-21d36210d724 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:03.348921 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74626f1-498a-43c8-9b5c-21d36210d724 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:03.348921 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74626f1-498a-43c8-9b5c-21d36210d724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:03.370256 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c74626f1-498a-43c8-9b5c-21d36210d724 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:41:03.371536 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c74626f1-498a-43c8-9b5c-21d36210d724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:41:03.460001 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 603/1221] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:41:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:41:03.751615 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42a264-ee04-41ee-9d4d-8f7f614fa72a None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:03.887561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42a264-ee04-41ee-9d4d-8f7f614fa72a None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:03.931077 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287f5c8-1731-42cc-93ff-13737fa77d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:03.931816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287f5c8-1731-42cc-93ff-13737fa77d98 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:03.932086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287f5c8-1731-42cc-93ff-13737fa77d98 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:03.932357 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287f5c8-1731-42cc-93ff-13737fa77d98 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:03.948472 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9287f5c8-1731-42cc-93ff-13737fa77d98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:41:03.954215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-9287f5c8-1731-42cc-93ff-13737fa77d98 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:41:03.954215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:03.954215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:03.954215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:03.954215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:03.954215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:03.954215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:03.954215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:03.954215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:03.954215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:03.954215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:03.954215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:03.954215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:41:03.954215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:41:03.954215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:41:03.954215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:41:03.955574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:41:03.955574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:41:03.955574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:41:03.955574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:41:03.955574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:03.955574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:03.955574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:03.955574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:03.955574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:03.955574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:03.955574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:03.955574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:03.955574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:03.955574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:03.955574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:03.955574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:03.956461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:03.956461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:03.956461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:03.956461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:03.956461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:41:03.956461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:41:03.956461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:41:03.956461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:41:03.956461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:41:03.956461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:41:03.956461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:41:03.956461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:41:03.956461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 619/1222] 10.222.0.71 () {56 vars in 1102 bytes} [Fri Aug 26 08:41:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:41:03.965832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74626f1-498a-43c8-9b5c-21d36210d724 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:04.109484 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74626f1-498a-43c8-9b5c-21d36210d724 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:04.143105 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:04.159529 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be06bf-fe70-4f1d-9a7d-834e2241efec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:04.160509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be06bf-fe70-4f1d-9a7d-834e2241efec None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:04.161035 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be06bf-fe70-4f1d-9a7d-834e2241efec None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:04.161509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be06bf-fe70-4f1d-9a7d-834e2241efec None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:04.163027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be06bf-fe70-4f1d-9a7d-834e2241efec None admin] RBAC: Adding query filter params (name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:04.163735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be06bf-fe70-4f1d-9a7d-834e2241efec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:04.166541 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be06bf-fe70-4f1d-9a7d-834e2241efec None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:04.178821 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 604/1223] 10.222.0.71 () {56 vars in 1103 bytes} [Fri Aug 26 08:41:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:04.185504 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287f5c8-1731-42cc-93ff-13737fa77d98 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:04.234766 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:04.300751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287f5c8-1731-42cc-93ff-13737fa77d98 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:04.344230 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2ae43-a6a0-4ebe-9c0c-59cf197f8fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:04.345579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2ae43-a6a0-4ebe-9c0c-59cf197f8fde None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:04.345579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2ae43-a6a0-4ebe-9c0c-59cf197f8fde None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:04.345579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2ae43-a6a0-4ebe-9c0c-59cf197f8fde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:04.358316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2ae43-a6a0-4ebe-9c0c-59cf197f8fde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:04.360235 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2ae43-a6a0-4ebe-9c0c-59cf197f8fde None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:04.380115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-8af2ae43-a6a0-4ebe-9c0c-59cf197f8fde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:41:04.380115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:04.380115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:04.380115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:04.380115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:04.380115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:04.380115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:04.380115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:04.380115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:04.380115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:04.380115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:04.380115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:04.380115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:41:04.380115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:41:04.380115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:41:04.380115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:41:04.381476 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:04.381476 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:04.381476 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:04.381476 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:04.381476 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:04.381476 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:04.381476 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:04.381476 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:04.381476 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:04.381476 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:04.381476 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:04.381476 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:04.381476 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:04.381476 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:04.381476 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:04.381476 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:04.381476 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:41:04.382720 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:41:04.382720 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:41:04.382720 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:41:04.382720 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 620/1224] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:41:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:41:04.389819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be06bf-fe70-4f1d-9a7d-834e2241efec None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:04.512055 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be06bf-fe70-4f1d-9a7d-834e2241efec None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:04.553390 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b53862-5ba0-485a-a4ed-f036fa43cd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:04.554049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b53862-5ba0-485a-a4ed-f036fa43cd44 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:04.554323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b53862-5ba0-485a-a4ed-f036fa43cd44 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:04.554594 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b53862-5ba0-485a-a4ed-f036fa43cd44 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:04.555373 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b53862-5ba0-485a-a4ed-f036fa43cd44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:04.555744 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b53862-5ba0-485a-a4ed-f036fa43cd44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:04.557207 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b53862-5ba0-485a-a4ed-f036fa43cd44 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:04.572657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 605/1225] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:04.579790 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2ae43-a6a0-4ebe-9c0c-59cf197f8fde None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:04.704387 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:04.712663 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2ae43-a6a0-4ebe-9c0c-59cf197f8fde None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:04.750232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6484d5-0a93-4b35-8ea5-e79638a03bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:04.750916 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6484d5-0a93-4b35-8ea5-e79638a03bb9 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:04.751202 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6484d5-0a93-4b35-8ea5-e79638a03bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:04.751480 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6484d5-0a93-4b35-8ea5-e79638a03bb9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:04.760804 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6484d5-0a93-4b35-8ea5-e79638a03bb9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:04.762244 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6484d5-0a93-4b35-8ea5-e79638a03bb9 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:04.780376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-fb6484d5-0a93-4b35-8ea5-e79638a03bb9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 08:41:04.780376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:04.780376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:04.780376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:04.780376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:04.780376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:04.780376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:04.780376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:04.780376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:04.780376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:04.780376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:04.780376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:04.780376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:41:04.780376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:41:04.780376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:41:04.780376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:41:04.782546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:04.782546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:04.782546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:41:04.782546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:41:04.782546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:41:04.782546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:41:04.782546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:04.782546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:04.782546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:04.782546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:04.782546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:04.782546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:04.782546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:04.782546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:04.782546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:04.782546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:04.782546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:04.782546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:04.784258 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:04.784258 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:04.784258 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:41:04.784258 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:41:04.784258 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:41:04.784258 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:41:04.784258 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:41:04.784258 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:41:04.784258 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 08:41:04.784258 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:41:04.784258 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 621/1226] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:41:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:41:04.788906 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b53862-5ba0-485a-a4ed-f036fa43cd44 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:04.847138 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:04.914300 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b53862-5ba0-485a-a4ed-f036fa43cd44 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:04.959678 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596da2eb-5966-4f75-bf0b-9343d196b90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:04.960245 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596da2eb-5966-4f75-bf0b-9343d196b90f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:04.960522 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596da2eb-5966-4f75-bf0b-9343d196b90f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:04.960804 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596da2eb-5966-4f75-bf0b-9343d196b90f None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:04.961546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596da2eb-5966-4f75-bf0b-9343d196b90f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:04.961850 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596da2eb-5966-4f75-bf0b-9343d196b90f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:04.963340 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596da2eb-5966-4f75-bf0b-9343d196b90f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:04.985855 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 606/1227] 10.222.0.71 () {56 vars in 1143 bytes} [Fri Aug 26 08:41:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:04.992145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6484d5-0a93-4b35-8ea5-e79638a03bb9 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:05.125165 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6484d5-0a93-4b35-8ea5-e79638a03bb9 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:05.185031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e673c-cc55-4e68-8027-2a2b813978c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:05.186298 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e673c-cc55-4e68-8027-2a2b813978c7 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:05.186656 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e673c-cc55-4e68-8027-2a2b813978c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:05.187159 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e673c-cc55-4e68-8027-2a2b813978c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:05.200677 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e673c-cc55-4e68-8027-2a2b813978c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:05.202214 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e673c-cc55-4e68-8027-2a2b813978c7 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:05.220761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-524e673c-cc55-4e68-8027-2a2b813978c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:41:05.220761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:05.220761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:05.220761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:05.220761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:05.220761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:05.220761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:05.220761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:05.220761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:05.220761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:05.220761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:05.220761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:05.220761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:41:05.220761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:41:05.220761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:41:05.220761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:41:05.222259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:05.222259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:05.222259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:05.222259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:05.222259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:05.222259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:05.222259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:05.222259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:05.222259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:05.222259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:05.222259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:05.222259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:05.222259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:05.222259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:05.222259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:05.222259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:05.222259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:41:05.224420 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:41:05.224420 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:41:05.224420 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:41:05.224420 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 622/1228] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:41:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:41:05.229795 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596da2eb-5966-4f75-bf0b-9343d196b90f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:05.365790 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596da2eb-5966-4f75-bf0b-9343d196b90f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:05.369978 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:05.417207 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3507a3e-1593-483c-9fa7-5d78b676c96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:05.417925 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3507a3e-1593-483c-9fa7-5d78b676c96e None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:05.418143 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3507a3e-1593-483c-9fa7-5d78b676c96e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:05.418590 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3507a3e-1593-483c-9fa7-5d78b676c96e None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:05.419229 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3507a3e-1593-483c-9fa7-5d78b676c96e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:05.419530 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3507a3e-1593-483c-9fa7-5d78b676c96e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:05.421459 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3507a3e-1593-483c-9fa7-5d78b676c96e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:05.436827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 607/1229] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:41:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:05.443899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e673c-cc55-4e68-8027-2a2b813978c7 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:05.458469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:05.586366 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e673c-cc55-4e68-8027-2a2b813978c7 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:05.647475 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75606218-1de0-4bb8-b241-185bd74c73b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:05.648267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75606218-1de0-4bb8-b241-185bd74c73b8 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:05.648543 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75606218-1de0-4bb8-b241-185bd74c73b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:05.648812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75606218-1de0-4bb8-b241-185bd74c73b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:05.657237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75606218-1de0-4bb8-b241-185bd74c73b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:05.659105 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75606218-1de0-4bb8-b241-185bd74c73b8 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:05.673551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-75606218-1de0-4bb8-b241-185bd74c73b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:41:05.673551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:05.673551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:05.673551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:05.673551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:05.673551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:05.673551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:05.673551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:05.673551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:05.673551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:05.673551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:05.673551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:05.673551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:41:05.673551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:41:05.673551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:41:05.674764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:41:05.674764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:05.674764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:05.674764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:05.674764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:05.674764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:05.674764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:05.674764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:05.674764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:05.674764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:05.674764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:05.674764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:05.674764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:05.674764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:05.674764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:05.674764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:05.674764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:05.675810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:41:05.675810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:41:05.675810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:41:05.675810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:41:05.675810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:41:05.675810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:41:05.675810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:41:05.675810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:41:05.675810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 623/1230] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:41:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:41:05.682172 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3507a3e-1593-483c-9fa7-5d78b676c96e None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:05.822004 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3507a3e-1593-483c-9fa7-5d78b676c96e None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:05.873710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c310a47-dd8d-4b9c-9cb7-ba71e3d3bff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:05.874410 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c310a47-dd8d-4b9c-9cb7-ba71e3d3bff2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:05.874720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c310a47-dd8d-4b9c-9cb7-ba71e3d3bff2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:05.875055 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c310a47-dd8d-4b9c-9cb7-ba71e3d3bff2 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:05.875850 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c310a47-dd8d-4b9c-9cb7-ba71e3d3bff2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:05.876250 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c310a47-dd8d-4b9c-9cb7-ba71e3d3bff2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:05.877738 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c310a47-dd8d-4b9c-9cb7-ba71e3d3bff2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:05.898502 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 608/1231] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:41:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:05.905785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75606218-1de0-4bb8-b241-185bd74c73b8 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:06.013556 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:06.070513 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75606218-1de0-4bb8-b241-185bd74c73b8 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:06.112564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777f0cd-5eda-4072-8ca0-59b3e4762af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:06.113302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777f0cd-5eda-4072-8ca0-59b3e4762af0 None admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:06.113654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777f0cd-5eda-4072-8ca0-59b3e4762af0 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:06.113847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777f0cd-5eda-4072-8ca0-59b3e4762af0 None admin] PATH_INFO: `/v3/projects/543d44293d894a7fbb83401f7a26dd9b/users/0cf5e2b4f5e440a7942adea3d51cec75/roles/33ac296b51c143e7bb44211175752edb` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:06.132769 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d777f0cd-5eda-4072-8ca0-59b3e4762af0 None admin] RBAC: Authorizing `identity:create_grant(project_id=543d44293d894a7fbb83401f7a26dd9b, user_id=0cf5e2b4f5e440a7942adea3d51cec75, role_id=33ac296b51c143e7bb44211175752edb)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:06.134456 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d777f0cd-5eda-4072-8ca0-59b3e4762af0 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:06.150954 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 624/1232] 10.222.0.71 () {58 vars in 1439 bytes} [Fri Aug 26 08:41:05 2022] PUT /identity/v3/projects/543d44293d894a7fbb83401f7a26dd9b/users/0cf5e2b4f5e440a7942adea3d51cec75/roles/33ac296b51c143e7bb44211175752edb => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:41:08.332470 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf39d9-8eac-4f4c-aa3c-40ea10a3d48f None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:08.333146 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf39d9-8eac-4f4c-aa3c-40ea10a3d48f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:08.334409 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf39d9-8eac-4f4c-aa3c-40ea10a3d48f None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:08.335291 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 609/1233] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:41:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:41:08.344476 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21b45f-4760-4d80-aef7-a5c8f85097a1 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:08.345396 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21b45f-4760-4d80-aef7-a5c8f85097a1 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:08.345960 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21b45f-4760-4d80-aef7-a5c8f85097a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:08.383953 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ab21b45f-4760-4d80-aef7-a5c8f85097a1 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:41:08.385563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ab21b45f-4760-4d80-aef7-a5c8f85097a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:41:08.463818 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 625/1234] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:41:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:41:08.755152 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf39d9-8eac-4f4c-aa3c-40ea10a3d48f None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:08.792462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:08.892870 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf39d9-8eac-4f4c-aa3c-40ea10a3d48f None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:08.947850 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4716b-89ef-4907-9e13-d1be8d2a3bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:08.948597 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4716b-89ef-4907-9e13-d1be8d2a3bb1 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:08.948882 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4716b-89ef-4907-9e13-d1be8d2a3bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:08.949179 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4716b-89ef-4907-9e13-d1be8d2a3bb1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:08.964258 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4716b-89ef-4907-9e13-d1be8d2a3bb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:41:08.971445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-26c4716b-89ef-4907-9e13-d1be8d2a3bb1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:41:08.971445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:08.971445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:08.971445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:08.971445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:08.971445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:08.971445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:08.971445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:08.971445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:08.971445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:08.971445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:08.971445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:08.971445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:41:08.971445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:41:08.971445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:41:08.971445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:41:08.972903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:41:08.972903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:41:08.972903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:41:08.972903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:41:08.972903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:08.972903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:08.972903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:08.972903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:08.972903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:08.972903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:08.972903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:08.972903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:08.972903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:08.972903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:08.972903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:08.972903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:08.973914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:08.973914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:08.973914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:08.973914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:08.973914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:41:08.973914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:41:08.973914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:41:08.973914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:41:08.973914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:41:08.973914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:41:08.973914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 08:41:08.973914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:41:08.979204 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 610/1235] 10.222.0.71 () {56 vars in 1102 bytes} [Fri Aug 26 08:41:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:41:08.980457 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21b45f-4760-4d80-aef7-a5c8f85097a1 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:09.042618 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:09.116140 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21b45f-4760-4d80-aef7-a5c8f85097a1 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:09.161839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099094c-cf25-4ab4-9241-72b03e0e5d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:09.162632 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099094c-cf25-4ab4-9241-72b03e0e5d7f None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:09.162984 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099094c-cf25-4ab4-9241-72b03e0e5d7f None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:09.163280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099094c-cf25-4ab4-9241-72b03e0e5d7f None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:09.164159 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c099094c-cf25-4ab4-9241-72b03e0e5d7f None admin] RBAC: Adding query filter params (name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:09.164479 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c099094c-cf25-4ab4-9241-72b03e0e5d7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:09.166185 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c099094c-cf25-4ab4-9241-72b03e0e5d7f None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:09.175828 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 626/1236] 10.222.0.71 () {56 vars in 1103 bytes} [Fri Aug 26 08:41:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:09.181876 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4716b-89ef-4907-9e13-d1be8d2a3bb1 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:09.297839 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:09.318292 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4716b-89ef-4907-9e13-d1be8d2a3bb1 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:09.357190 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe549b9e-e168-4b22-a9f1-1c04ffbc6661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:09.358719 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe549b9e-e168-4b22-a9f1-1c04ffbc6661 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:09.359203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe549b9e-e168-4b22-a9f1-1c04ffbc6661 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:09.359480 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe549b9e-e168-4b22-a9f1-1c04ffbc6661 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:09.371819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe549b9e-e168-4b22-a9f1-1c04ffbc6661 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:09.373296 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe549b9e-e168-4b22-a9f1-1c04ffbc6661 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:09.389865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-fe549b9e-e168-4b22-a9f1-1c04ffbc6661 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:41:09.389865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:09.389865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:09.389865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:09.389865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:09.389865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:09.389865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:09.389865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:09.389865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:09.389865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:09.389865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:09.389865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:09.389865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:41:09.389865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:41:09.389865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:41:09.389865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:41:09.391171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:09.391171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:09.391171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:09.391171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:09.391171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:09.391171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:09.391171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:09.391171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:09.391171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:09.391171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:09.391171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:09.391171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:09.391171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:09.391171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:09.391171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:09.391171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:09.391171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:41:09.397355 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:41:09.397355 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:41:09.397355 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:41:09.397355 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 611/1237] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:41:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:41:09.398383 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099094c-cf25-4ab4-9241-72b03e0e5d7f None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:09.524700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099094c-cf25-4ab4-9241-72b03e0e5d7f None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:09.534494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:09.567044 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf5c28c-3338-4d81-a44d-f1eaf36c6a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:09.567619 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5c28c-3338-4d81-a44d-f1eaf36c6a68 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:09.568146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5c28c-3338-4d81-a44d-f1eaf36c6a68 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:09.568498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf5c28c-3338-4d81-a44d-f1eaf36c6a68 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:09.569505 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf5c28c-3338-4d81-a44d-f1eaf36c6a68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:09.569868 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf5c28c-3338-4d81-a44d-f1eaf36c6a68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:09.571578 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf5c28c-3338-4d81-a44d-f1eaf36c6a68 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:09.589813 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 627/1238] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:41:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:09.597645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe549b9e-e168-4b22-a9f1-1c04ffbc6661 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:09.720942 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe549b9e-e168-4b22-a9f1-1c04ffbc6661 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:09.769827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7fede-6b53-4af0-9a43-f6e7a0957482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:09.770477 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7fede-6b53-4af0-9a43-f6e7a0957482 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:09.770757 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7fede-6b53-4af0-9a43-f6e7a0957482 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:09.771065 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7fede-6b53-4af0-9a43-f6e7a0957482 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:09.773971 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:09.782041 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7fede-6b53-4af0-9a43-f6e7a0957482 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:09.784318 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7fede-6b53-4af0-9a43-f6e7a0957482 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:09.798357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-c8c7fede-6b53-4af0-9a43-f6e7a0957482 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 08:41:09.798357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:09.798357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:09.798357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:09.798357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:09.798357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:09.798357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:09.798357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:09.798357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:09.798357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:09.798357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:09.798357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:09.798357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:41:09.798357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:41:09.798357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:41:09.798357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:41:09.799664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:09.799664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:09.799664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:41:09.799664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:41:09.799664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:41:09.799664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:41:09.799664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:09.799664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:09.799664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:09.799664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:09.799664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:09.799664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:09.799664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:09.799664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:09.799664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:09.799664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:09.799664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:09.799664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:09.806086 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:09.806086 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:09.806086 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:41:09.806086 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:41:09.806086 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:41:09.806086 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:41:09.806086 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:41:09.806086 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:41:09.806086 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 08:41:09.806086 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:41:09.806086 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 612/1239] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:41:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:41:09.808119 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf5c28c-3338-4d81-a44d-f1eaf36c6a68 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:09.935165 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf5c28c-3338-4d81-a44d-f1eaf36c6a68 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:09.972410 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273b6db-36e4-44ab-8d1a-18636f897413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:09.973069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273b6db-36e4-44ab-8d1a-18636f897413 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:09.973356 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273b6db-36e4-44ab-8d1a-18636f897413 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:09.973702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273b6db-36e4-44ab-8d1a-18636f897413 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:09.974447 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b273b6db-36e4-44ab-8d1a-18636f897413 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:09.974745 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b273b6db-36e4-44ab-8d1a-18636f897413 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:09.976406 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b273b6db-36e4-44ab-8d1a-18636f897413 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:10.003318 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 628/1240] 10.222.0.71 () {56 vars in 1143 bytes} [Fri Aug 26 08:41:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:10.010009 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7fede-6b53-4af0-9a43-f6e7a0957482 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:10.129375 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7fede-6b53-4af0-9a43-f6e7a0957482 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:10.169642 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5af8c-5fd9-43a0-8b37-ad1e5841523f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:10.170505 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5af8c-5fd9-43a0-8b37-ad1e5841523f None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:10.170777 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5af8c-5fd9-43a0-8b37-ad1e5841523f None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:10.171073 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5af8c-5fd9-43a0-8b37-ad1e5841523f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:10.242582 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:10.250865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5af8c-5fd9-43a0-8b37-ad1e5841523f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:10.252626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b5af8c-5fd9-43a0-8b37-ad1e5841523f None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:10.389295 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 613/1241] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:41:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 08:41:10.395712 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273b6db-36e4-44ab-8d1a-18636f897413 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:10.430217 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:10.526853 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273b6db-36e4-44ab-8d1a-18636f897413 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:10.568586 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d0b4b-465b-47bf-b35b-10c9aae410d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:10.569286 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d0b4b-465b-47bf-b35b-10c9aae410d2 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:10.569563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d0b4b-465b-47bf-b35b-10c9aae410d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:10.569871 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d0b4b-465b-47bf-b35b-10c9aae410d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:10.586075 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d0b4b-465b-47bf-b35b-10c9aae410d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:10.587733 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d0b4b-465b-47bf-b35b-10c9aae410d2 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:10.604786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-7c1d0b4b-465b-47bf-b35b-10c9aae410d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:41:10.604786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:10.604786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:10.604786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:10.604786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:10.604786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:10.604786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:10.604786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:10.604786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:10.604786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:10.604786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:10.604786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:10.604786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:41:10.604786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:41:10.604786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:41:10.604786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:41:10.606641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:10.606641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:10.606641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:10.606641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:10.606641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:10.606641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:10.606641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:10.606641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:10.606641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:10.606641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:10.606641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:10.606641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:10.606641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:10.606641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:10.606641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:10.606641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:10.606641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:41:10.608037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:41:10.608037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:41:10.608037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:41:10.608037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 629/1242] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:41:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:41:10.613468 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5af8c-5fd9-43a0-8b37-ad1e5841523f None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:10.736387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:10.743565 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b5af8c-5fd9-43a0-8b37-ad1e5841523f None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:10.786507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1daa5ab-e43b-464f-9178-5890dbfd5328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:10.787133 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1daa5ab-e43b-464f-9178-5890dbfd5328 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:10.787400 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1daa5ab-e43b-464f-9178-5890dbfd5328 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:10.787670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1daa5ab-e43b-464f-9178-5890dbfd5328 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:10.788403 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1daa5ab-e43b-464f-9178-5890dbfd5328 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:10.788724 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1daa5ab-e43b-464f-9178-5890dbfd5328 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:10.790255 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1daa5ab-e43b-464f-9178-5890dbfd5328 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:10.804793 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 614/1243] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:41:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:10.811184 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d0b4b-465b-47bf-b35b-10c9aae410d2 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:10.908621 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:10.936705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d0b4b-465b-47bf-b35b-10c9aae410d2 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:10.980093 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68480d-7a5e-4cc7-b879-b8af75fe311e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:10.981188 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68480d-7a5e-4cc7-b879-b8af75fe311e None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:10.981459 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68480d-7a5e-4cc7-b879-b8af75fe311e None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:10.981762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68480d-7a5e-4cc7-b879-b8af75fe311e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:10.989447 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68480d-7a5e-4cc7-b879-b8af75fe311e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:10.991031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68480d-7a5e-4cc7-b879-b8af75fe311e None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:11.006116 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-fc68480d-7a5e-4cc7-b879-b8af75fe311e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:41:11.006116 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:11.006116 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:11.006116 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:11.006116 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:11.006116 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:11.006116 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:11.006116 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:11.006116 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:11.006116 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:11.006116 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:11.006116 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:11.006116 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:41:11.006116 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:41:11.006116 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:41:11.007354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:41:11.007354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:11.007354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:11.007354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:11.007354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:11.007354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:11.007354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:11.007354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:11.007354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:11.007354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:11.007354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:11.007354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:11.007354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:11.007354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:11.007354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:11.007354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:11.007354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:11.008299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:41:11.008299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:41:11.008299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:41:11.008299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:41:11.008299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:41:11.008299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:41:11.008299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:41:11.008299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:41:11.008299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 630/1244] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:41:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:41:11.013664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1daa5ab-e43b-464f-9178-5890dbfd5328 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:11.144864 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1daa5ab-e43b-464f-9178-5890dbfd5328 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:11.182682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5448e-0ba0-4731-a5fe-5537746dec81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:11.183339 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5448e-0ba0-4731-a5fe-5537746dec81 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:11.183701 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5448e-0ba0-4731-a5fe-5537746dec81 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:11.183990 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5448e-0ba0-4731-a5fe-5537746dec81 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:11.184694 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba5448e-0ba0-4731-a5fe-5537746dec81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:11.185002 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba5448e-0ba0-4731-a5fe-5537746dec81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:11.186456 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba5448e-0ba0-4731-a5fe-5537746dec81 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:11.201788 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 615/1245] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:41:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:11.208676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68480d-7a5e-4cc7-b879-b8af75fe311e None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:11.335732 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68480d-7a5e-4cc7-b879-b8af75fe311e None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:11.385502 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a829da3-a72f-421e-84e2-b07b3d1f2548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:11.386262 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a829da3-a72f-421e-84e2-b07b3d1f2548 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:11.386575 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a829da3-a72f-421e-84e2-b07b3d1f2548 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:11.386820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a829da3-a72f-421e-84e2-b07b3d1f2548 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:11.387698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a829da3-a72f-421e-84e2-b07b3d1f2548 None admin] RBAC: Adding query filter params (role.id=33ac296b51c143e7bb44211175752edb, scope.project.id=543d44293d894a7fbb83401f7a26dd9b, user.id=0cf5e2b4f5e440a7942adea3d51cec75) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:11.388041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a829da3-a72f-421e-84e2-b07b3d1f2548 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:11.389570 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a829da3-a72f-421e-84e2-b07b3d1f2548 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:11.396883 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 631/1246] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:41:11 2022] GET /identity/v3/role_assignments?user.id=0cf5e2b4f5e440a7942adea3d51cec75&scope.project.id=543d44293d894a7fbb83401f7a26dd9b&role.id=33ac296b51c143e7bb44211175752edb => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:41:13.490937 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ae4fe-bd82-459a-8ccd-895ea8d38f6e None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:13.491484 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ae4fe-bd82-459a-8ccd-895ea8d38f6e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:13.491484 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ae4fe-bd82-459a-8ccd-895ea8d38f6e None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:13.493265 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 616/1247] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:41:13.499238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f530c-3368-4c21-b682-38e760556a4e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:13.499695 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f530c-3368-4c21-b682-38e760556a4e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:13.499787 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f530c-3368-4c21-b682-38e760556a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:13.507350 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:13.522696 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-0a8f530c-3368-4c21-b682-38e760556a4e None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:41:13.524284 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-0a8f530c-3368-4c21-b682-38e760556a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:41:13.612214 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 632/1248] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:41:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:41:13.896375 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ae4fe-bd82-459a-8ccd-895ea8d38f6e None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:13.920026 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:14.030133 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ae4fe-bd82-459a-8ccd-895ea8d38f6e None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:14.071905 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b5ecb-2b43-42d0-adc4-727f9ebce8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:14.072896 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b5ecb-2b43-42d0-adc4-727f9ebce8f3 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:14.073213 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b5ecb-2b43-42d0-adc4-727f9ebce8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:14.073489 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b5ecb-2b43-42d0-adc4-727f9ebce8f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:14.086337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b5ecb-2b43-42d0-adc4-727f9ebce8f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:41:14.092709 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-237b5ecb-2b43-42d0-adc4-727f9ebce8f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:41:14.092709 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:14.092709 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:14.092709 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:14.092709 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:14.092709 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:14.092709 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:14.092709 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:14.092709 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:14.092709 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:14.092709 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:14.092709 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:14.092709 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:41:14.092709 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:41:14.092709 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:41:14.092709 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:41:14.093720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:41:14.093720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:41:14.093720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:41:14.093720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:41:14.093720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:14.093720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:14.093720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:14.093720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:14.093720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:14.093720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:14.093720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:14.093720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:14.093720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:14.093720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:14.093720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:14.093720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:14.094696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:14.094696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:14.094696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:14.094696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:14.094696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:41:14.094696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:41:14.094696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:41:14.094696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:41:14.094696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:41:14.094696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:41:14.094696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:41:14.094696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:41:14.094696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 617/1249] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:41:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:41:14.104019 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f530c-3368-4c21-b682-38e760556a4e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:14.217323 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f530c-3368-4c21-b682-38e760556a4e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:14.252444 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:14.258760 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59548fa-7271-4d99-b715-5e32007aea06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:14.259527 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59548fa-7271-4d99-b715-5e32007aea06 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:14.259808 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59548fa-7271-4d99-b715-5e32007aea06 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:14.260099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59548fa-7271-4d99-b715-5e32007aea06 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:14.260887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59548fa-7271-4d99-b715-5e32007aea06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:14.261192 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59548fa-7271-4d99-b715-5e32007aea06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:14.262798 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59548fa-7271-4d99-b715-5e32007aea06 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:14.277391 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 633/1250] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:41:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:14.279119 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b5ecb-2b43-42d0-adc4-727f9ebce8f3 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:14.376540 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:14.407097 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b5ecb-2b43-42d0-adc4-727f9ebce8f3 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:14.450279 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd9561-27d2-498e-9b80-b853ff7e74e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:14.451072 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd9561-27d2-498e-9b80-b853ff7e74e4 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:14.451406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd9561-27d2-498e-9b80-b853ff7e74e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:14.451729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd9561-27d2-498e-9b80-b853ff7e74e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:14.467763 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd9561-27d2-498e-9b80-b853ff7e74e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:14.469478 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd9561-27d2-498e-9b80-b853ff7e74e4 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:14.486618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-b0bd9561-27d2-498e-9b80-b853ff7e74e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:41:14.486618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:14.486618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:14.486618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:14.486618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:14.486618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:14.486618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:14.486618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:14.486618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:14.486618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:14.486618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:14.486618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:14.486618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:41:14.486618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:41:14.486618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:41:14.486618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:41:14.487925 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:14.487925 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:14.487925 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:14.487925 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:14.487925 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:14.487925 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:14.487925 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:14.487925 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:14.487925 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:14.487925 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:14.487925 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:14.487925 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:14.487925 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:14.487925 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:14.487925 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:14.487925 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:14.487925 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:41:14.488986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:41:14.488986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:41:14.488986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:41:14.488986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 618/1251] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:41:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:41:14.494228 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59548fa-7271-4d99-b715-5e32007aea06 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:14.624487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59548fa-7271-4d99-b715-5e32007aea06 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:14.664502 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee9253-0cfc-4e08-8661-45f641f5370a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:14.665046 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee9253-0cfc-4e08-8661-45f641f5370a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:14.665383 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee9253-0cfc-4e08-8661-45f641f5370a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:14.665644 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee9253-0cfc-4e08-8661-45f641f5370a None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:14.666332 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee9253-0cfc-4e08-8661-45f641f5370a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:14.666637 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee9253-0cfc-4e08-8661-45f641f5370a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:14.668109 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee9253-0cfc-4e08-8661-45f641f5370a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:14.686653 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 634/1252] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:41:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:14.688090 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd9561-27d2-498e-9b80-b853ff7e74e4 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:14.813300 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd9561-27d2-498e-9b80-b853ff7e74e4 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:14.845903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:14.857973 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b644b5-6b2c-4357-adef-e66516d5c510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:14.859131 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b644b5-6b2c-4357-adef-e66516d5c510 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:14.859687 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b644b5-6b2c-4357-adef-e66516d5c510 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:14.860197 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b644b5-6b2c-4357-adef-e66516d5c510 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:14.873603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b644b5-6b2c-4357-adef-e66516d5c510 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:14.876423 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b644b5-6b2c-4357-adef-e66516d5c510 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:14.895374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-96b644b5-6b2c-4357-adef-e66516d5c510 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 08:41:14.895374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:14.895374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:14.895374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:14.895374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:14.895374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:14.895374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:14.895374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:14.895374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:14.895374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:14.895374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:14.895374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:14.895374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:41:14.895374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:41:14.895374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:41:14.895374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:41:14.896462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:14.896462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:14.896462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:41:14.896462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:41:14.896462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:41:14.896462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:41:14.896462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:14.896462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:14.896462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:14.896462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:14.896462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:14.896462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:14.896462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:14.896462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:14.896462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:14.896462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:14.896462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:14.896462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:14.897656 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:14.897656 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:14.897656 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:41:14.897656 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:41:14.897656 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:41:14.897656 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:41:14.897656 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:41:14.897656 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:41:14.897656 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 08:41:14.897656 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:41:14.897656 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 619/1253] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:41:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:41:14.904834 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee9253-0cfc-4e08-8661-45f641f5370a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:14.974906 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:15.032981 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee9253-0cfc-4e08-8661-45f641f5370a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:15.074605 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c821381-a00d-428e-94c8-6cba7821e711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:15.075200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c821381-a00d-428e-94c8-6cba7821e711 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:15.075521 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c821381-a00d-428e-94c8-6cba7821e711 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:15.075821 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c821381-a00d-428e-94c8-6cba7821e711 None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:15.076569 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c821381-a00d-428e-94c8-6cba7821e711 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:15.076906 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c821381-a00d-428e-94c8-6cba7821e711 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:15.078376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c821381-a00d-428e-94c8-6cba7821e711 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:15.101632 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 635/1254] 10.222.0.71 () {56 vars in 1143 bytes} [Fri Aug 26 08:41:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:15.108214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b644b5-6b2c-4357-adef-e66516d5c510 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:15.235930 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b644b5-6b2c-4357-adef-e66516d5c510 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:15.280563 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f230aab-ee19-44b4-ab62-460c21cbdf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:15.281421 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f230aab-ee19-44b4-ab62-460c21cbdf18 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:15.281771 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f230aab-ee19-44b4-ab62-460c21cbdf18 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:15.282167 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f230aab-ee19-44b4-ab62-460c21cbdf18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:15.353260 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:15.371287 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f230aab-ee19-44b4-ab62-460c21cbdf18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:15.372942 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f230aab-ee19-44b4-ab62-460c21cbdf18 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:15.500853 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 620/1255] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:41:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 08:41:15.510683 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c821381-a00d-428e-94c8-6cba7821e711 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:15.661321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c821381-a00d-428e-94c8-6cba7821e711 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:15.677272 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:15.702041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d80629-2028-4d4f-9b37-57592edc3156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:15.703058 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d80629-2028-4d4f-9b37-57592edc3156 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:15.703264 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d80629-2028-4d4f-9b37-57592edc3156 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:15.703630 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d80629-2028-4d4f-9b37-57592edc3156 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:15.717029 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d80629-2028-4d4f-9b37-57592edc3156 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:15.718320 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d80629-2028-4d4f-9b37-57592edc3156 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:15.738208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-64d80629-2028-4d4f-9b37-57592edc3156 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:41:15.738208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:15.738208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:15.738208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:15.738208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:15.738208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:15.738208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:15.738208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:15.738208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:15.738208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:15.738208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:15.738208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:15.738208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:41:15.738208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:41:15.738208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:41:15.738208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:41:15.739705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:15.739705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:15.739705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:15.739705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:15.739705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:15.739705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:15.739705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:15.739705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:15.739705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:15.739705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:15.739705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:15.739705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:15.739705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:15.739705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:15.739705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:15.739705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:15.739705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:41:15.740675 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:41:15.740675 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:41:15.740675 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:41:15.740675 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 636/1256] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:41:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:41:15.746626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f230aab-ee19-44b4-ab62-460c21cbdf18 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:15.875601 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:15.890532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f230aab-ee19-44b4-ab62-460c21cbdf18 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:15.937074 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd3731-5927-4194-906b-b4bf1bf68bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:15.937697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd3731-5927-4194-906b-b4bf1bf68bf9 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:15.938071 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd3731-5927-4194-906b-b4bf1bf68bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:15.938377 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd3731-5927-4194-906b-b4bf1bf68bf9 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:15.939151 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd3731-5927-4194-906b-b4bf1bf68bf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:15.939457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd3731-5927-4194-906b-b4bf1bf68bf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:15.940918 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd3731-5927-4194-906b-b4bf1bf68bf9 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:15.956758 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 621/1257] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:41:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:41:15.962486 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d80629-2028-4d4f-9b37-57592edc3156 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:16.115934 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d80629-2028-4d4f-9b37-57592edc3156 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:16.172672 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a264b75-59f1-46d0-a887-89ef83b81f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:16.173394 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a264b75-59f1-46d0-a887-89ef83b81f30 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:16.173676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a264b75-59f1-46d0-a887-89ef83b81f30 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:16.173949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a264b75-59f1-46d0-a887-89ef83b81f30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:16.185104 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a264b75-59f1-46d0-a887-89ef83b81f30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:16.187880 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a264b75-59f1-46d0-a887-89ef83b81f30 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:16.206663 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-5a264b75-59f1-46d0-a887-89ef83b81f30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:41:16.206663 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:16.206663 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:16.206663 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:16.206663 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:16.206663 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:16.206663 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:16.206663 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:16.206663 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:16.206663 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:16.206663 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:16.206663 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:16.206663 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:41:16.206663 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:41:16.206663 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:41:16.207744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:41:16.207744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:16.207744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:16.207744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:16.207744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:16.207744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:16.207744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:16.207744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:16.207744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:16.207744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:16.207744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:16.207744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:16.207744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:16.207744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:16.207744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:16.207744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:16.207744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:16.209039 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:41:16.209039 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:41:16.209039 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:41:16.209039 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:41:16.209039 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:41:16.209039 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:41:16.209039 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:41:16.209039 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:41:16.209039 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 637/1258] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:41:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:41:16.214014 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd3731-5927-4194-906b-b4bf1bf68bf9 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:16.331395 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd3731-5927-4194-906b-b4bf1bf68bf9 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:16.373595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7737e8-c3df-4355-9388-ebb82aff6b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:16.374222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7737e8-c3df-4355-9388-ebb82aff6b92 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:16.374492 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7737e8-c3df-4355-9388-ebb82aff6b92 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:16.374762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7737e8-c3df-4355-9388-ebb82aff6b92 None admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:16.375547 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7737e8-c3df-4355-9388-ebb82aff6b92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:16.375851 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7737e8-c3df-4355-9388-ebb82aff6b92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:16.377305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7737e8-c3df-4355-9388-ebb82aff6b92 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:16.399321 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 622/1259] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:41:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:16.400636 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a264b75-59f1-46d0-a887-89ef83b81f30 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:16.439553 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:16.526558 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a264b75-59f1-46d0-a887-89ef83b81f30 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:16.566704 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b7202-ac37-4df8-a85f-a4524472fb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:16.567592 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b7202-ac37-4df8-a85f-a4524472fb7c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:16.567883 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b7202-ac37-4df8-a85f-a4524472fb7c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:16.568232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b7202-ac37-4df8-a85f-a4524472fb7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:16.568995 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518b7202-ac37-4df8-a85f-a4524472fb7c None admin] RBAC: Adding query filter params (role.id=18c3c8a821b647f7982a955880bd0a9a, scope.project.id=543d44293d894a7fbb83401f7a26dd9b, user.id=0cf5e2b4f5e440a7942adea3d51cec75) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:16.569422 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518b7202-ac37-4df8-a85f-a4524472fb7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:16.571331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518b7202-ac37-4df8-a85f-a4524472fb7c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:16.581063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 638/1260] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:41:16 2022] GET /identity/v3/role_assignments?user.id=0cf5e2b4f5e440a7942adea3d51cec75&scope.project.id=543d44293d894a7fbb83401f7a26dd9b&role.id=18c3c8a821b647f7982a955880bd0a9a => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:18.658412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f04b5-dc64-443c-aa54-f349e5838b03 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:18.659048 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f04b5-dc64-443c-aa54-f349e5838b03 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:18.659048 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f04b5-dc64-443c-aa54-f349e5838b03 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:18.661001 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 623/1261] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:41:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:41:18.668228 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067e42e-edbd-4c70-b705-27ada5db8b0e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:18.668889 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067e42e-edbd-4c70-b705-27ada5db8b0e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:18.668889 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067e42e-edbd-4c70-b705-27ada5db8b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:18.690561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-3067e42e-edbd-4c70-b705-27ada5db8b0e None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:41:18.692006 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3067e42e-edbd-4c70-b705-27ada5db8b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:41:18.769578 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 639/1262] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:41:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:41:19.035968 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f04b5-dc64-443c-aa54-f349e5838b03 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:19.049200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:19.152647 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f04b5-dc64-443c-aa54-f349e5838b03 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:19.192355 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b5147-03c0-4760-bc39-c751ebd94dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:19.192997 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b5147-03c0-4760-bc39-c751ebd94dce None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:19.193273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b5147-03c0-4760-bc39-c751ebd94dce None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:19.193542 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b5147-03c0-4760-bc39-c751ebd94dce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:19.205165 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b5147-03c0-4760-bc39-c751ebd94dce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:41:19.210007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-e54b5147-03c0-4760-bc39-c751ebd94dce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:41:19.210007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:19.210007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:19.210007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:19.210007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:19.210007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:19.210007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:19.210007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:19.210007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:19.210007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:19.210007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:19.210007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:19.210007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:41:19.210007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:41:19.210007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:41:19.210007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:41:19.215159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:41:19.215159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:41:19.215159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:41:19.215159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:41:19.215159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:19.215159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:19.215159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:19.215159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:19.215159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:19.215159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:19.215159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:19.215159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:19.215159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:19.215159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:19.215159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:19.215159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:19.216259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:19.216259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:19.216259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:19.216259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:19.216259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:41:19.216259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:41:19.216259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:41:19.216259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:41:19.216259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:41:19.216259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:41:19.216259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:41:19.216259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:41:19.216259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 624/1263] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:41:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:41:19.219935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067e42e-edbd-4c70-b705-27ada5db8b0e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:19.242623 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:19.352116 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067e42e-edbd-4c70-b705-27ada5db8b0e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:19.392590 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1371f261-1477-44e7-8e7d-2f80447ac1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:19.393220 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371f261-1477-44e7-8e7d-2f80447ac1f3 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:19.393497 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371f261-1477-44e7-8e7d-2f80447ac1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:19.393765 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371f261-1477-44e7-8e7d-2f80447ac1f3 None admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:19.394448 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1371f261-1477-44e7-8e7d-2f80447ac1f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:19.394747 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1371f261-1477-44e7-8e7d-2f80447ac1f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:19.396179 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1371f261-1477-44e7-8e7d-2f80447ac1f3 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:19.405828 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 640/1264] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:41:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:19.412947 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b5147-03c0-4760-bc39-c751ebd94dce None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:19.491596 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:19.532915 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b5147-03c0-4760-bc39-c751ebd94dce None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:19.574096 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3825004-1577-411a-ab90-ce5137faca5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:19.574795 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3825004-1577-411a-ab90-ce5137faca5b None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:19.575087 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3825004-1577-411a-ab90-ce5137faca5b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:19.575357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3825004-1577-411a-ab90-ce5137faca5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:19.588199 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3825004-1577-411a-ab90-ce5137faca5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:19.589984 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3825004-1577-411a-ab90-ce5137faca5b None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:19.608208 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-e3825004-1577-411a-ab90-ce5137faca5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:41:19.608208 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:19.608208 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:19.608208 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:19.608208 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:19.608208 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:19.608208 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:19.608208 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:19.608208 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:19.608208 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:19.608208 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:19.608208 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:19.608208 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:41:19.608208 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:41:19.608208 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:41:19.608208 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:41:19.609297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:19.609297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:19.609297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:19.609297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:19.609297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:19.609297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:19.609297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:19.609297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:19.609297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:19.609297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:19.609297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:19.609297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:19.609297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:19.609297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:19.609297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:19.609297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:19.609297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:41:19.610326 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:41:19.610326 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:41:19.610326 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:41:19.610326 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 625/1265] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:41:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:41:19.616981 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1371f261-1477-44e7-8e7d-2f80447ac1f3 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:19.709208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:19.730541 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1371f261-1477-44e7-8e7d-2f80447ac1f3 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:19.771216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ddffe-1683-494b-9e31-5cd6bacb3b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:19.771832 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ddffe-1683-494b-9e31-5cd6bacb3b71 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:19.772104 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ddffe-1683-494b-9e31-5cd6bacb3b71 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:19.772371 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ddffe-1683-494b-9e31-5cd6bacb3b71 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:19.773021 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ddffe-1683-494b-9e31-5cd6bacb3b71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:19.773318 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ddffe-1683-494b-9e31-5cd6bacb3b71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:19.774701 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ddffe-1683-494b-9e31-5cd6bacb3b71 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:19.787131 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 641/1266] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:41:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:19.793112 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3825004-1577-411a-ab90-ce5137faca5b None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:19.905774 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3825004-1577-411a-ab90-ce5137faca5b None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:19.927268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:19.949665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf10c3-907e-44e1-aa3b-242241ea923b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:19.950338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf10c3-907e-44e1-aa3b-242241ea923b None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:19.950607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf10c3-907e-44e1-aa3b-242241ea923b None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:19.950868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf10c3-907e-44e1-aa3b-242241ea923b None admin] PATH_INFO: `/v3/users/placement` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:19.960387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bf10c3-907e-44e1-aa3b-242241ea923b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:19.961849 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bf10c3-907e-44e1-aa3b-242241ea923b None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:19.975358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-59bf10c3-907e-44e1-aa3b-242241ea923b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 08:41:19.975358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:19.975358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:19.975358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:19.975358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:19.975358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:19.975358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:19.975358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:19.975358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:19.975358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:19.975358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:19.975358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:19.975358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:41:19.975358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:41:19.975358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:41:19.975358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:41:19.976607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:19.976607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:19.976607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:41:19.976607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:41:19.976607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:41:19.976607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:41:19.976607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:19.976607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:19.976607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:19.976607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:19.976607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:19.976607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:19.976607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:19.976607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:19.976607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:19.976607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:19.976607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:19.976607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:19.977992 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:19.977992 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:19.977992 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:41:19.977992 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:41:19.977992 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:41:19.977992 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:41:19.977992 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:41:19.977992 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:41:19.977992 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 08:41:19.977992 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:41:19.977992 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 626/1267] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:41:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:41:19.983117 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ddffe-1683-494b-9e31-5cd6bacb3b71 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:20.118803 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ddffe-1683-494b-9e31-5cd6bacb3b71 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:20.162495 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e551fa6-4eff-40b9-9695-ec334d86af0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:20.163091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e551fa6-4eff-40b9-9695-ec334d86af0a None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:20.163374 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e551fa6-4eff-40b9-9695-ec334d86af0a None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:20.163643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e551fa6-4eff-40b9-9695-ec334d86af0a None admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:20.164418 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e551fa6-4eff-40b9-9695-ec334d86af0a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:20.164714 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e551fa6-4eff-40b9-9695-ec334d86af0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:20.166197 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e551fa6-4eff-40b9-9695-ec334d86af0a None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:20.178953 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:20.191421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 642/1268] 10.222.0.71 () {56 vars in 1143 bytes} [Fri Aug 26 08:41:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:20.198113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf10c3-907e-44e1-aa3b-242241ea923b None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:20.314967 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf10c3-907e-44e1-aa3b-242241ea923b None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:20.369497 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c06ca-3c84-4338-a358-f04f134b39ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:20.370165 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c06ca-3c84-4338-a358-f04f134b39ba None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:20.370451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c06ca-3c84-4338-a358-f04f134b39ba None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:20.370724 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c06ca-3c84-4338-a358-f04f134b39ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:20.385256 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385c06ca-3c84-4338-a358-f04f134b39ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:20.386743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385c06ca-3c84-4338-a358-f04f134b39ba None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:20.403370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-385c06ca-3c84-4338-a358-f04f134b39ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:41:20.403370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:20.403370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:20.403370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:20.403370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:20.403370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:20.403370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:20.403370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:20.403370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:20.403370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:20.403370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:20.403370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:20.403370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:41:20.403370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:41:20.403370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:41:20.403370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:41:20.404773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:20.404773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:20.404773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:20.404773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:20.404773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:20.404773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:20.404773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:20.404773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:20.404773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:20.404773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:20.404773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:20.404773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:20.404773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:20.404773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:20.404773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:20.404773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:20.404773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:41:20.405810 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:41:20.405810 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:41:20.405810 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:41:20.405810 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 627/1269] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:41:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:41:20.411014 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e551fa6-4eff-40b9-9695-ec334d86af0a None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:20.536996 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e551fa6-4eff-40b9-9695-ec334d86af0a None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:20.575784 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5123db6-2417-4022-b572-f5b370e57164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:20.576695 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5123db6-2417-4022-b572-f5b370e57164 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:20.577466 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5123db6-2417-4022-b572-f5b370e57164 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:20.577466 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5123db6-2417-4022-b572-f5b370e57164 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:20.578212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5123db6-2417-4022-b572-f5b370e57164 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:20.578597 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5123db6-2417-4022-b572-f5b370e57164 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:20.580219 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5123db6-2417-4022-b572-f5b370e57164 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:20.596615 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 643/1270] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:20.603519 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c06ca-3c84-4338-a358-f04f134b39ba None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:20.650173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:20.772104 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c06ca-3c84-4338-a358-f04f134b39ba None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:20.830198 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61973cdf-9b98-4e1b-9763-e1642d936727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:20.831011 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61973cdf-9b98-4e1b-9763-e1642d936727 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:20.831398 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61973cdf-9b98-4e1b-9763-e1642d936727 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:20.831770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61973cdf-9b98-4e1b-9763-e1642d936727 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:20.842798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61973cdf-9b98-4e1b-9763-e1642d936727 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:20.845055 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61973cdf-9b98-4e1b-9763-e1642d936727 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:20.863593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-61973cdf-9b98-4e1b-9763-e1642d936727 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:41:20.863593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:20.863593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:20.863593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:20.863593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:20.863593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:20.863593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:20.863593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:20.863593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:20.863593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:20.863593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:20.863593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:20.863593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:41:20.863593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:41:20.863593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:41:20.864682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:41:20.864682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:20.864682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:20.864682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:20.864682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:20.864682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:20.864682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:20.864682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:20.864682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:20.864682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:20.864682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:20.864682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:20.864682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:20.864682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:20.864682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:20.864682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:20.864682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:20.865566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:41:20.865566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:41:20.865566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:41:20.865566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:41:20.865566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:41:20.865566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:41:20.865566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:41:20.865566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:41:20.865566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 628/1271] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:41:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:41:20.871833 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5123db6-2417-4022-b572-f5b370e57164 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:20.939254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:20.984664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5123db6-2417-4022-b572-f5b370e57164 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:21.021020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1ddcf-0079-4cf6-90ee-d05979c8350d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:21.021651 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1ddcf-0079-4cf6-90ee-d05979c8350d None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:21.021927 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1ddcf-0079-4cf6-90ee-d05979c8350d None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:21.022198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1ddcf-0079-4cf6-90ee-d05979c8350d None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:21.022915 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da1ddcf-0079-4cf6-90ee-d05979c8350d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:21.023231 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da1ddcf-0079-4cf6-90ee-d05979c8350d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:21.024677 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da1ddcf-0079-4cf6-90ee-d05979c8350d None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:21.038557 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 644/1272] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:41:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:21.045073 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61973cdf-9b98-4e1b-9763-e1642d936727 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:21.150776 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:21.161679 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61973cdf-9b98-4e1b-9763-e1642d936727 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:21.202645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e6399-903f-4c1a-92c1-3eea75a8cce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:21.203381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e6399-903f-4c1a-92c1-3eea75a8cce3 None admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:21.203666 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e6399-903f-4c1a-92c1-3eea75a8cce3 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:21.203946 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e6399-903f-4c1a-92c1-3eea75a8cce3 None admin] PATH_INFO: `/v3/projects/543d44293d894a7fbb83401f7a26dd9b/users/0cf5e2b4f5e440a7942adea3d51cec75/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:21.206044 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e6399-903f-4c1a-92c1-3eea75a8cce3 None admin] RBAC: Authorizing `identity:create_grant(project_id=543d44293d894a7fbb83401f7a26dd9b, user_id=0cf5e2b4f5e440a7942adea3d51cec75, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:21.207588 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e6399-903f-4c1a-92c1-3eea75a8cce3 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:21.224338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 629/1273] 10.222.0.71 () {58 vars in 1439 bytes} [Fri Aug 26 08:41:21 2022] PUT /identity/v3/projects/543d44293d894a7fbb83401f7a26dd9b/users/0cf5e2b4f5e440a7942adea3d51cec75/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:41:23.184188 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8f873-8843-431f-9902-48cbab00bc70 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:23.184765 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8f873-8843-431f-9902-48cbab00bc70 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:23.184765 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8f873-8843-431f-9902-48cbab00bc70 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:23.186063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 645/1274] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:41:23.192027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcf2e2-dd3a-402b-aa80-c58763c4adb9 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:23.192519 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcf2e2-dd3a-402b-aa80-c58763c4adb9 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:23.192616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcf2e2-dd3a-402b-aa80-c58763c4adb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:23.219145 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-34dcf2e2-dd3a-402b-aa80-c58763c4adb9 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:41:23.220494 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-34dcf2e2-dd3a-402b-aa80-c58763c4adb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:41:23.298054 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 630/1275] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:41:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 08:41:23.562938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8f873-8843-431f-9902-48cbab00bc70 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:23.711237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8f873-8843-431f-9902-48cbab00bc70 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:23.730876 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:23.750704 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a25c9-4db4-4fa0-b981-2da4c8e3ecb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:23.751357 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a25c9-4db4-4fa0-b981-2da4c8e3ecb1 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:23.751632 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a25c9-4db4-4fa0-b981-2da4c8e3ecb1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:23.751903 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a25c9-4db4-4fa0-b981-2da4c8e3ecb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:23.763229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a25c9-4db4-4fa0-b981-2da4c8e3ecb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 08:41:23.768248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-196a25c9-4db4-4fa0-b981-2da4c8e3ecb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:41:23.768248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:23.768248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:23.768248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:23.768248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:23.768248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:23.768248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:23.768248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:23.768248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:23.768248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:23.768248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:23.768248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:23.768248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 08:41:23.768248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 08:41:23.768248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 08:41:23.768248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 08:41:23.769390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 08:41:23.769390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 08:41:23.769390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 08:41:23.769390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 08:41:23.769390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:23.769390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:23.769390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:23.769390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:23.769390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:23.769390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:23.769390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:23.769390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:23.769390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:23.769390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:23.769390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:23.769390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:23.770467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:23.770467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:23.770467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:23.770467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:23.770467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 08:41:23.770467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 08:41:23.770467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 08:41:23.770467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 08:41:23.770467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 08:41:23.770467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 08:41:23.770467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 08:41:23.770467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:41:23.770467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 646/1276] 10.222.0.71 () {56 vars in 1096 bytes} [Fri Aug 26 08:41:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:41:23.779058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dcf2e2-dd3a-402b-aa80-c58763c4adb9 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:23.870280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:23.892154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dcf2e2-dd3a-402b-aa80-c58763c4adb9 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:23.931605 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4f2de-19cc-43a6-8d8b-3c85351e14ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:23.932223 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4f2de-19cc-43a6-8d8b-3c85351e14ae None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:23.932501 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4f2de-19cc-43a6-8d8b-3c85351e14ae None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:23.932772 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4f2de-19cc-43a6-8d8b-3c85351e14ae None admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:23.933467 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4f2de-19cc-43a6-8d8b-3c85351e14ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:23.933803 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4f2de-19cc-43a6-8d8b-3c85351e14ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:23.935271 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4f2de-19cc-43a6-8d8b-3c85351e14ae None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:23.944914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 631/1277] 10.222.0.71 () {56 vars in 1099 bytes} [Fri Aug 26 08:41:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:23.951509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a25c9-4db4-4fa0-b981-2da4c8e3ecb1 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:24.070136 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a25c9-4db4-4fa0-b981-2da4c8e3ecb1 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:24.106673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a07c1-a421-4b46-b444-55803eec4a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:24.107365 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a07c1-a421-4b46-b444-55803eec4a07 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:24.107637 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a07c1-a421-4b46-b444-55803eec4a07 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:24.107908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a07c1-a421-4b46-b444-55803eec4a07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:24.119152 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a07c1-a421-4b46-b444-55803eec4a07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:24.120755 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a07c1-a421-4b46-b444-55803eec4a07 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:24.136837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-b49a07c1-a421-4b46-b444-55803eec4a07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:41:24.136837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:24.136837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:24.136837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:24.136837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:24.136837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:24.136837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:24.136837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:24.136837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:24.136837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:24.136837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:24.136837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:24.136837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:41:24.136837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:41:24.136837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:41:24.136837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:41:24.138091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:24.138091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:24.138091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:24.138091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:24.138091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:24.138091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:24.138091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:24.138091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:24.138091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:24.138091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:24.138091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:24.138091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:24.138091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:24.138091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:24.138091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:24.138091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:24.138091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:41:24.139147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:41:24.139147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:41:24.139147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:41:24.139147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 647/1278] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:41:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:41:24.144353 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4f2de-19cc-43a6-8d8b-3c85351e14ae None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:24.256094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4f2de-19cc-43a6-8d8b-3c85351e14ae None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:24.293109 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21046f86-d4ef-4798-894e-0624e9a8bd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:24.293727 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21046f86-d4ef-4798-894e-0624e9a8bd13 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:24.294004 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21046f86-d4ef-4798-894e-0624e9a8bd13 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:24.294269 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21046f86-d4ef-4798-894e-0624e9a8bd13 None admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:24.294969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21046f86-d4ef-4798-894e-0624e9a8bd13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:24.295268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21046f86-d4ef-4798-894e-0624e9a8bd13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:24.296649 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21046f86-d4ef-4798-894e-0624e9a8bd13 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:24.309339 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 632/1279] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:24.317475 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a07c1-a421-4b46-b444-55803eec4a07 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:24.352107 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:24.429035 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a07c1-a421-4b46-b444-55803eec4a07 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:24.468392 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f7b22-8984-4975-8450-fc7c2e81f6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:24.469049 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f7b22-8984-4975-8450-fc7c2e81f6f3 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:24.469350 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f7b22-8984-4975-8450-fc7c2e81f6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:24.469632 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f7b22-8984-4975-8450-fc7c2e81f6f3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:24.484235 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4f7b22-8984-4975-8450-fc7c2e81f6f3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:24.485848 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4f7b22-8984-4975-8450-fc7c2e81f6f3 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:24.497957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-5c4f7b22-8984-4975-8450-fc7c2e81f6f3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 08:41:24.497957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:24.497957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:24.497957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:24.497957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:24.497957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:24.497957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:24.497957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:24.497957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:24.497957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:24.497957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:24.497957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:24.497957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 08:41:24.497957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 08:41:24.497957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 08:41:24.497957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 08:41:24.499299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:24.499299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:24.499299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 08:41:24.499299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:41:24.499299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 08:41:24.499299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 08:41:24.499299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:24.499299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:24.499299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:24.499299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:24.499299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:24.499299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:24.499299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:24.499299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:24.499299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:24.499299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:24.499299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:24.499299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:24.500292 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:24.500292 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:24.500292 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 08:41:24.500292 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 08:41:24.500292 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 08:41:24.500292 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 08:41:24.500292 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 08:41:24.500292 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 08:41:24.500292 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 08:41:24.500292 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:41:24.500292 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 648/1280] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:41:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:41:24.505642 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21046f86-d4ef-4798-894e-0624e9a8bd13 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:24.511642 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:24.629670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21046f86-d4ef-4798-894e-0624e9a8bd13 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:24.670137 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24847821-b3a0-49b7-880b-edfdae6894f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:24.670942 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24847821-b3a0-49b7-880b-edfdae6894f6 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:24.671380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24847821-b3a0-49b7-880b-edfdae6894f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:24.671655 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24847821-b3a0-49b7-880b-edfdae6894f6 None admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:24.672645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24847821-b3a0-49b7-880b-edfdae6894f6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:24.672948 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24847821-b3a0-49b7-880b-edfdae6894f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:24.674416 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24847821-b3a0-49b7-880b-edfdae6894f6 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:24.696908 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 633/1281] 10.222.0.71 () {56 vars in 1143 bytes} [Fri Aug 26 08:41:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:24.703657 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f7b22-8984-4975-8450-fc7c2e81f6f3 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:24.818599 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:24.830419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f7b22-8984-4975-8450-fc7c2e81f6f3 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:24.869605 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d301a-3d8b-4a01-834d-9f9fcf791653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:24.870280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d301a-3d8b-4a01-834d-9f9fcf791653 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:24.870558 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d301a-3d8b-4a01-834d-9f9fcf791653 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:24.870833 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d301a-3d8b-4a01-834d-9f9fcf791653 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:24.950071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d301a-3d8b-4a01-834d-9f9fcf791653 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:24.951613 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d301a-3d8b-4a01-834d-9f9fcf791653 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:25.055784 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:25.066904 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 649/1282] 10.222.0.71 () {58 vars in 1282 bytes} [Fri Aug 26 08:41:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 08:41:25.073329 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24847821-b3a0-49b7-880b-edfdae6894f6 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:25.153052 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:25.196861 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24847821-b3a0-49b7-880b-edfdae6894f6 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:25.235502 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983790d7-43ca-4a00-ae98-cd0756dc4430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:25.236079 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983790d7-43ca-4a00-ae98-cd0756dc4430 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:25.236363 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983790d7-43ca-4a00-ae98-cd0756dc4430 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:25.236629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983790d7-43ca-4a00-ae98-cd0756dc4430 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:25.249261 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983790d7-43ca-4a00-ae98-cd0756dc4430 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:25.250861 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983790d7-43ca-4a00-ae98-cd0756dc4430 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:25.267920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-983790d7-43ca-4a00-ae98-cd0756dc4430 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:41:25.267920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:25.267920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:25.267920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:25.267920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:25.267920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:25.267920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:25.267920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:25.267920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:25.267920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:25.267920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:25.267920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:25.267920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 08:41:25.267920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 08:41:25.267920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 08:41:25.267920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 08:41:25.269633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:25.269633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:25.269633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:25.269633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:25.269633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:25.269633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:25.269633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:25.269633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:25.269633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:25.269633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:25.269633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:25.269633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:25.269633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:25.269633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:25.269633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:25.269633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:25.269633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 08:41:25.270806 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 08:41:25.270806 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 08:41:25.270806 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:41:25.270806 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 634/1283] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:41:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:41:25.275329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d301a-3d8b-4a01-834d-9f9fcf791653 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:25.382677 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d301a-3d8b-4a01-834d-9f9fcf791653 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:25.420508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24569a-6db4-4771-85f5-eca58d589793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:25.421313 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24569a-6db4-4771-85f5-eca58d589793 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:25.421619 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24569a-6db4-4771-85f5-eca58d589793 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:25.421619 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24569a-6db4-4771-85f5-eca58d589793 None admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:25.422319 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b24569a-6db4-4771-85f5-eca58d589793 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:25.422652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b24569a-6db4-4771-85f5-eca58d589793 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:25.424085 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b24569a-6db4-4771-85f5-eca58d589793 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:25.436737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 650/1284] 10.222.0.71 () {56 vars in 1109 bytes} [Fri Aug 26 08:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:25.442960 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983790d7-43ca-4a00-ae98-cd0756dc4430 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:25.560065 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983790d7-43ca-4a00-ae98-cd0756dc4430 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:25.576587 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:25.605528 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ae8d4-0a22-441d-8daa-e8d02feb5923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:25.606176 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ae8d4-0a22-441d-8daa-e8d02feb5923 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:25.606450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ae8d4-0a22-441d-8daa-e8d02feb5923 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:25.606721 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ae8d4-0a22-441d-8daa-e8d02feb5923 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:25.614700 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ae8d4-0a22-441d-8daa-e8d02feb5923 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:25.616262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ae8d4-0a22-441d-8daa-e8d02feb5923 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:25.629280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application [None req-bc4ae8d4-0a22-441d-8daa-e8d02feb5923 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:41:25.629280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:25.629280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:25.629280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:25.629280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:25.629280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:25.629280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:25.629280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:25.629280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:25.629280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:25.629280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:25.629280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:25.629280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 08:41:25.629280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 08:41:25.629280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 08:41:25.630880 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 08:41:25.630880 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:25.630880 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:25.630880 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:25.630880 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:25.630880 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:25.630880 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:25.630880 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:25.630880 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:25.630880 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:25.630880 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:25.630880 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:25.630880 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:25.630880 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:25.630880 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:25.630880 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:25.630880 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:25.632203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 08:41:25.632203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 08:41:25.632203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 08:41:25.632203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 08:41:25.632203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 08:41:25.632203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 08:41:25.632203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 08:41:25.632203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: ERROR keystone.server.flask.application Aug 26 08:41:25.632203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 635/1285] 10.222.0.71 () {56 vars in 1111 bytes} [Fri Aug 26 08:41:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:41:25.637478 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24569a-6db4-4771-85f5-eca58d589793 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:25.704914 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:25.770722 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24569a-6db4-4771-85f5-eca58d589793 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:25.805602 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c102c4-a06d-4e81-9f3a-e066647111b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:25.806237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c102c4-a06d-4e81-9f3a-e066647111b1 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:25.806520 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c102c4-a06d-4e81-9f3a-e066647111b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:25.806795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c102c4-a06d-4e81-9f3a-e066647111b1 None admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:25.807637 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c102c4-a06d-4e81-9f3a-e066647111b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:25.807894 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c102c4-a06d-4e81-9f3a-e066647111b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:25.809376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c102c4-a06d-4e81-9f3a-e066647111b1 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:25.822232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 651/1286] 10.222.0.71 () {56 vars in 1148 bytes} [Fri Aug 26 08:41:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:25.829352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ae8d4-0a22-441d-8daa-e8d02feb5923 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:25.964115 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ae8d4-0a22-441d-8daa-e8d02feb5923 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:26.003369 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db559b9e-6433-4cd7-b9b4-4e828ac47e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:26.003983 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db559b9e-6433-4cd7-b9b4-4e828ac47e06 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:26.004304 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db559b9e-6433-4cd7-b9b4-4e828ac47e06 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:26.004901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db559b9e-6433-4cd7-b9b4-4e828ac47e06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:26.005659 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db559b9e-6433-4cd7-b9b4-4e828ac47e06 None admin] RBAC: Adding query filter params (role.id=18c3c8a821b647f7982a955880bd0a9a, scope.project.id=543d44293d894a7fbb83401f7a26dd9b, user.id=0cf5e2b4f5e440a7942adea3d51cec75) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:26.005965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db559b9e-6433-4cd7-b9b4-4e828ac47e06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:26.007486 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db559b9e-6433-4cd7-b9b4-4e828ac47e06 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:26.014812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 636/1287] 10.222.0.71 () {56 vars in 1374 bytes} [Fri Aug 26 08:41:25 2022] GET /identity/v3/role_assignments?user.id=0cf5e2b4f5e440a7942adea3d51cec75&scope.project.id=543d44293d894a7fbb83401f7a26dd9b&role.id=18c3c8a821b647f7982a955880bd0a9a => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:27.996735 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4597db-1f87-49a4-a4dd-37d636452a4e None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:27.997314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4597db-1f87-49a4-a4dd-37d636452a4e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:27.997314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4597db-1f87-49a4-a4dd-37d636452a4e None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:27.998704 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 652/1288] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:41:28.005447 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217daf9c-877d-41a7-a858-f6f275b34d44 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:28.006045 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217daf9c-877d-41a7-a858-f6f275b34d44 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:28.006045 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217daf9c-877d-41a7-a858-f6f275b34d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:28.026574 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-217daf9c-877d-41a7-a858-f6f275b34d44 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:41:28.027895 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-217daf9c-877d-41a7-a858-f6f275b34d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:41:28.067981 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:28.112630 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 637/1289] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:41:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:41:28.385659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4597db-1f87-49a4-a4dd-37d636452a4e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:28.495788 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4597db-1f87-49a4-a4dd-37d636452a4e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:28.507703 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:28.535709 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b2332-cc5e-41fa-be40-29d5b7f0de5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:28.536375 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b2332-cc5e-41fa-be40-29d5b7f0de5f None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:28.536618 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b2332-cc5e-41fa-be40-29d5b7f0de5f None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:28.536888 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b2332-cc5e-41fa-be40-29d5b7f0de5f None admin] PATH_INFO: `/v3/services/placement` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:28.537497 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b2332-cc5e-41fa-be40-29d5b7f0de5f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:28.538968 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b2332-cc5e-41fa-be40-29d5b7f0de5f None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:28.548198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-454b2332-cc5e-41fa-be40-29d5b7f0de5f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 08:41:28.548198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:28.548198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 08:41:28.548198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 08:41:28.548198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 08:41:28.548198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 08:41:28.548198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 08:41:28.548198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:41:28.548198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 08:41:28.548198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:41:28.548198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:41:28.548198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:41:28.548198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:28.548198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:28.548198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:28.548198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:28.548198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:28.549559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:28.549559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:28.549559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:28.549559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:28.549559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:28.549559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:28.549559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 08:41:28.549559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 08:41:28.549559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 08:41:28.549559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 08:41:28.549559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:28.549559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:28.549559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:28.549559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:28.549559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:28.549559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:28.550583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:28.550583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:28.550583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:28.550583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:28.550583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:28.550583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:28.550583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:28.550583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:28.550583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:28.550583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:28.550583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 08:41:28.550583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:41:28.550583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 08:41:28.550583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:41:28.550583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 653/1290] 10.222.0.71 () {56 vars in 1117 bytes} [Fri Aug 26 08:41:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:41:28.557024 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217daf9c-877d-41a7-a858-f6f275b34d44 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:28.667937 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217daf9c-877d-41a7-a858-f6f275b34d44 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:28.704920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13435753-4817-44d7-898b-ac1f73580d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:28.705536 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13435753-4817-44d7-898b-ac1f73580d6e None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:28.705811 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13435753-4817-44d7-898b-ac1f73580d6e None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:28.706087 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13435753-4817-44d7-898b-ac1f73580d6e None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:28.706723 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13435753-4817-44d7-898b-ac1f73580d6e None admin] RBAC: Adding query filter params (name=placement) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:28.707045 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13435753-4817-44d7-898b-ac1f73580d6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:28.708420 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13435753-4817-44d7-898b-ac1f73580d6e None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:28.713727 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 638/1291] 10.222.0.71 () {56 vars in 1116 bytes} [Fri Aug 26 08:41:28 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:28.721099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b2332-cc5e-41fa-be40-29d5b7f0de5f None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:28.831467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b2332-cc5e-41fa-be40-29d5b7f0de5f None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:28.867277 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb487e6-fda9-4377-9f57-0d8039077f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:28.867887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb487e6-fda9-4377-9f57-0d8039077f97 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:28.868166 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb487e6-fda9-4377-9f57-0d8039077f97 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:28.868433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb487e6-fda9-4377-9f57-0d8039077f97 None admin] PATH_INFO: `/v3/services` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:28.869072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb487e6-fda9-4377-9f57-0d8039077f97 None admin] RBAC: Adding query filter params (type=placement) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:28.869378 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb487e6-fda9-4377-9f57-0d8039077f97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:28.870768 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb487e6-fda9-4377-9f57-0d8039077f97 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:28.875912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 654/1292] 10.222.0.71 () {56 vars in 1116 bytes} [Fri Aug 26 08:41:28 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:30.776745 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e43f40-6122-493d-a5d7-cac576db6fca None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:30.777335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e43f40-6122-493d-a5d7-cac576db6fca None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:30.777335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e43f40-6122-493d-a5d7-cac576db6fca None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:30.779348 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 639/1293] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:41:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:41:30.787756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6b6dd-e82a-4a84-aa63-053ba62dd071 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:30.788471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6b6dd-e82a-4a84-aa63-053ba62dd071 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:30.788471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6b6dd-e82a-4a84-aa63-053ba62dd071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:30.810044 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-fff6b6dd-e82a-4a84-aa63-053ba62dd071 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:41:30.811401 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-fff6b6dd-e82a-4a84-aa63-053ba62dd071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:41:30.843480 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:30.883796 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 655/1294] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:41:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:41:31.151785 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e43f40-6122-493d-a5d7-cac576db6fca None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:31.186507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:31.267599 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e43f40-6122-493d-a5d7-cac576db6fca None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:31.302660 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f066a6-b749-4d9e-98ba-a547904430d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:31.303317 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f066a6-b749-4d9e-98ba-a547904430d8 None admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:31.303593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f066a6-b749-4d9e-98ba-a547904430d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:31.303863 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f066a6-b749-4d9e-98ba-a547904430d8 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:31.304519 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f066a6-b749-4d9e-98ba-a547904430d8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:31.305893 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f066a6-b749-4d9e-98ba-a547904430d8 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:31.322714 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 640/1295] 10.222.0.71 () {60 vars in 1141 bytes} [Fri Aug 26 08:41:31 2022] POST /identity/v3/services => generated 246 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:41:33.237814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d3f78-b9f1-4439-8985-b603f9f72f1a None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:33.238426 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d3f78-b9f1-4439-8985-b603f9f72f1a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:33.238426 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d3f78-b9f1-4439-8985-b603f9f72f1a None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:33.239728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 656/1296] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:41:33.245256 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a7afa-3661-4ca2-bfcc-28434349d458 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:33.246010 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a7afa-3661-4ca2-bfcc-28434349d458 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:33.246010 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a7afa-3661-4ca2-bfcc-28434349d458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:33.265478 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-030a7afa-3661-4ca2-bfcc-28434349d458 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:41:33.266861 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-030a7afa-3661-4ca2-bfcc-28434349d458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:41:33.342996 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 641/1297] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:41:33 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:41:33.617694 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d3f78-b9f1-4439-8985-b603f9f72f1a None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:33.748311 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d3f78-b9f1-4439-8985-b603f9f72f1a None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:33.786298 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903032aa-654d-4f7d-8100-70e2c0f58d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:33.786935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903032aa-654d-4f7d-8100-70e2c0f58d86 None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:33.787220 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903032aa-654d-4f7d-8100-70e2c0f58d86 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:33.787482 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903032aa-654d-4f7d-8100-70e2c0f58d86 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:33.788071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903032aa-654d-4f7d-8100-70e2c0f58d86 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:33.789465 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903032aa-654d-4f7d-8100-70e2c0f58d86 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:33.798676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-903032aa-654d-4f7d-8100-70e2c0f58d86 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 08:41:33.798676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:33.798676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 08:41:33.798676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 08:41:33.798676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 08:41:33.798676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 08:41:33.798676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 08:41:33.798676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:41:33.798676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 08:41:33.798676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:41:33.798676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:41:33.798676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:41:33.798676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:33.798676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:33.798676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:33.798676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:33.798676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:33.799925 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:33.799925 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:33.799925 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:33.799925 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:33.799925 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:33.799925 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:33.799925 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 08:41:33.799925 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 08:41:33.799925 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 08:41:33.799925 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 08:41:33.799925 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:33.799925 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:33.799925 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:33.799925 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:33.799925 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:33.799925 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:33.800795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:33.800795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:33.800795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:33.800795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:33.800795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:33.800795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:33.800795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:33.800795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:33.800795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:33.800795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:33.800795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 08:41:33.800795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:41:33.800795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 08:41:33.800795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:41:33.800795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 657/1298] 10.222.0.71 () {56 vars in 1117 bytes} [Fri Aug 26 08:41:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 08:41:33.806010 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a7afa-3661-4ca2-bfcc-28434349d458 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:33.825056 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:33.912841 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a7afa-3661-4ca2-bfcc-28434349d458 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:33.949163 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be80385-1ca0-438b-80ea-69b24dff04e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:33.949764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be80385-1ca0-438b-80ea-69b24dff04e2 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:33.950037 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be80385-1ca0-438b-80ea-69b24dff04e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:33.950301 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be80385-1ca0-438b-80ea-69b24dff04e2 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:33.950938 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be80385-1ca0-438b-80ea-69b24dff04e2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:33.951251 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be80385-1ca0-438b-80ea-69b24dff04e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:33.952628 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be80385-1ca0-438b-80ea-69b24dff04e2 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:33.958230 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 642/1299] 10.222.0.71 () {56 vars in 1116 bytes} [Fri Aug 26 08:41:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:33.964522 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903032aa-654d-4f7d-8100-70e2c0f58d86 None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:33.997686 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:34.071076 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903032aa-654d-4f7d-8100-70e2c0f58d86 None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:34.106512 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c53f1a-893f-4509-9a88-563499e23e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:34.107154 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c53f1a-893f-4509-9a88-563499e23e8c None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:34.107430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c53f1a-893f-4509-9a88-563499e23e8c None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:34.107699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c53f1a-893f-4509-9a88-563499e23e8c None admin] PATH_INFO: `/v3/endpoints` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:34.108407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c53f1a-893f-4509-9a88-563499e23e8c None admin] RBAC: Adding query filter params (interface=public, service_id=b39082bf64a24b0386e938c39d48a72d, region_id=RegionOne) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:34.108711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c53f1a-893f-4509-9a88-563499e23e8c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:34.110102 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c53f1a-893f-4509-9a88-563499e23e8c None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:34.116425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 658/1300] 10.222.0.71 () {56 vars in 1251 bytes} [Fri Aug 26 08:41:33 2022] GET /identity/v3/endpoints?service_id=b39082bf64a24b0386e938c39d48a72d&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:34.122870 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be80385-1ca0-438b-80ea-69b24dff04e2 None admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:34.228900 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:34.249642 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be80385-1ca0-438b-80ea-69b24dff04e2 None admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:34.286004 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94541852-0bdf-4a45-b7c7-115bd7c243a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:34.286581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94541852-0bdf-4a45-b7c7-115bd7c243a3 None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:34.286921 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94541852-0bdf-4a45-b7c7-115bd7c243a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:34.287190 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94541852-0bdf-4a45-b7c7-115bd7c243a3 None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:34.287800 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94541852-0bdf-4a45-b7c7-115bd7c243a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:34.289154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94541852-0bdf-4a45-b7c7-115bd7c243a3 None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:34.296321 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 643/1301] 10.222.0.71 () {56 vars in 1087 bytes} [Fri Aug 26 08:41:34 2022] GET /identity/v3/services => generated 2191 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:41:36.234220 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887db97-d8a4-4394-9075-b1b1a4b57026 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:36.234859 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887db97-d8a4-4394-9075-b1b1a4b57026 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:36.235026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887db97-d8a4-4394-9075-b1b1a4b57026 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:36.236792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 659/1302] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:41:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:41:36.242697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45579084-32cd-4e1b-8763-a1012181252b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:36.243205 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45579084-32cd-4e1b-8763-a1012181252b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:36.243374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45579084-32cd-4e1b-8763-a1012181252b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:36.264512 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-45579084-32cd-4e1b-8763-a1012181252b None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:41:36.265890 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-45579084-32cd-4e1b-8763-a1012181252b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:41:36.364762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 644/1303] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:41:36 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 08:41:36.690545 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d887db97-d8a4-4394-9075-b1b1a4b57026 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:36.804778 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:36.813154 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d887db97-d8a4-4394-9075-b1b1a4b57026 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:36.851897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba7215-697c-4b37-9abe-ddbf6394c6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:36.852541 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba7215-697c-4b37-9abe-ddbf6394c6ec None admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:36.852816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba7215-697c-4b37-9abe-ddbf6394c6ec None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:36.853086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba7215-697c-4b37-9abe-ddbf6394c6ec None admin] PATH_INFO: `/v3/services/placement` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:36.853682 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba7215-697c-4b37-9abe-ddbf6394c6ec None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:36.855098 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba7215-697c-4b37-9abe-ddbf6394c6ec None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:36.864546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application [None req-06ba7215-697c-4b37-9abe-ddbf6394c6ec None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 08:41:36.864546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:36.864546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 08:41:36.864546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 08:41:36.864546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 08:41:36.864546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 08:41:36.864546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 08:41:36.864546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:41:36.864546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 08:41:36.864546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:41:36.864546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 08:41:36.864546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:41:36.864546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 08:41:36.864546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 08:41:36.864546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 08:41:36.864546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 08:41:36.864546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 08:41:36.866241 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 08:41:36.866241 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 08:41:36.866241 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 08:41:36.866241 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 08:41:36.866241 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 08:41:36.866241 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 08:41:36.866241 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 08:41:36.866241 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 08:41:36.866241 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 08:41:36.866241 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 08:41:36.866241 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 08:41:36.866241 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 08:41:36.866241 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 08:41:36.866241 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 08:41:36.866241 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 08:41:36.866241 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 08:41:36.867249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 08:41:36.867249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application with Lock( Aug 26 08:41:36.867249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 08:41:36.867249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self._enter() Aug 26 08:41:36.867249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 08:41:36.867249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 08:41:36.867249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 08:41:36.867249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application return self.creator() Aug 26 08:41:36.867249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 08:41:36.867249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application created_value = creator( Aug 26 08:41:36.867249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 08:41:36.867249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 08:41:36.867249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 08:41:36.867249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: ERROR keystone.server.flask.application Aug 26 08:41:36.867249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 660/1304] 10.222.0.71 () {56 vars in 1117 bytes} [Fri Aug 26 08:41:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 08:41:36.876630 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45579084-32cd-4e1b-8763-a1012181252b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:36.961084 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:36.987098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45579084-32cd-4e1b-8763-a1012181252b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:37.024771 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13976c32-6e0d-4a4c-80cb-3eee697db3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:37.025391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13976c32-6e0d-4a4c-80cb-3eee697db3ef None admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:37.025657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13976c32-6e0d-4a4c-80cb-3eee697db3ef None admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:37.025936 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13976c32-6e0d-4a4c-80cb-3eee697db3ef None admin] PATH_INFO: `/v3/services` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:37.026573 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13976c32-6e0d-4a4c-80cb-3eee697db3ef None admin] RBAC: Adding query filter params (name=placement) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:41:37.026873 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13976c32-6e0d-4a4c-80cb-3eee697db3ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:37.028327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13976c32-6e0d-4a4c-80cb-3eee697db3ef None admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:37.033903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 645/1305] 10.222.0.71 () {56 vars in 1116 bytes} [Fri Aug 26 08:41:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:41:37.040013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba7215-697c-4b37-9abe-ddbf6394c6ec None admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:41:37.159292 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba7215-697c-4b37-9abe-ddbf6394c6ec None admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:41:37.199561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d512b-af16-493b-b37b-a10c72407f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:41:37.200179 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d512b-af16-493b-b37b-a10c72407f65 None admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:41:37.200461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d512b-af16-493b-b37b-a10c72407f65 None admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:41:37.200731 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d512b-af16-493b-b37b-a10c72407f65 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:41:37.201399 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d512b-af16-493b-b37b-a10c72407f65 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:41:37.202786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d512b-af16-493b-b37b-a10c72407f65 None admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:41:37.224792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 661/1306] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:41:37 2022] POST /identity/v3/endpoints => generated 331 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:41:47.030770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=30 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:41:47.220378 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:42:49.586838 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac33ae6-470b-48ba-8b51-0e2cf5ff6f7a None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:42:49.587676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac33ae6-470b-48ba-8b51-0e2cf5ff6f7a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:42:49.587817 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac33ae6-470b-48ba-8b51-0e2cf5ff6f7a None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:42:49.591352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 646/1307] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:42:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:42:49.600100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10e8a4-a023-4b30-9a4b-c86ed9837cfe None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:42:49.600815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10e8a4-a023-4b30-9a4b-c86ed9837cfe None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:42:49.601153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10e8a4-a023-4b30-9a4b-c86ed9837cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:42:49.907048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-bb10e8a4-a023-4b30-9a4b-c86ed9837cfe None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:42:49.908406 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-bb10e8a4-a023-4b30-9a4b-c86ed9837cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:42:50.146163 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 662/1308] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 548 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:42:50.589395 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9164a81-6866-423a-b487-5c8c45629420 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:42:50.589955 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9164a81-6866-423a-b487-5c8c45629420 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:42:50.589955 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9164a81-6866-423a-b487-5c8c45629420 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:42:50.593192 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 647/1309] 10.222.0.71 () {54 vars in 965 bytes} [Fri Aug 26 08:42:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:42:50.601117 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40509579-2c22-401e-867a-61de009854af None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:42:50.601823 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40509579-2c22-401e-867a-61de009854af None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:42:50.602113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40509579-2c22-401e-867a-61de009854af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:42:50.730784 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-40509579-2c22-401e-867a-61de009854af None None] MFA Rules not processed for user `4aa8938a504c4c4f8815478e520482d8`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:42:50.732080 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-40509579-2c22-401e-867a-61de009854af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:42:50.772223 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:42:50.875438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 663/1310] 10.222.0.71 () {58 vars in 1062 bytes} [Fri Aug 26 08:42:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:42:50.886758 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9164a81-6866-423a-b487-5c8c45629420 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:42:50.889980 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e9164a81-6866-423a-b487-5c8c45629420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:42:51.067655 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9164a81-6866-423a-b487-5c8c45629420 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:42:51.198739 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e99641-88ea-4449-9c36-27256fdca18a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:42:51.199455 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e99641-88ea-4449-9c36-27256fdca18a service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:42:51.199836 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e99641-88ea-4449-9c36-27256fdca18a service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:42:51.200131 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e99641-88ea-4449-9c36-27256fdca18a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:42:51.202869 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-19e99641-88ea-4449-9c36-27256fdca18a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:42:51.304609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:42:51.422508 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e99641-88ea-4449-9c36-27256fdca18a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:42:51.424886 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e99641-88ea-4449-9c36-27256fdca18a service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:42:51.456055 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 648/1311] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:42:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:42:54.111519 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d73f85-c16a-4821-afa6-4f3317782f38 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:42:54.112089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d73f85-c16a-4821-afa6-4f3317782f38 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:42:54.112280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d73f85-c16a-4821-afa6-4f3317782f38 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:42:54.115756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 664/1312] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:42:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:42:54.120458 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add35935-d301-42cc-b6c7-4e4e373bd3c1 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:42:54.121210 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add35935-d301-42cc-b6c7-4e4e373bd3c1 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:42:54.121210 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add35935-d301-42cc-b6c7-4e4e373bd3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:42:54.201028 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-add35935-d301-42cc-b6c7-4e4e373bd3c1 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:42:54.202238 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-add35935-d301-42cc-b6c7-4e4e373bd3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:42:54.341432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 649/1313] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:42:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:42:54.622930 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d73f85-c16a-4821-afa6-4f3317782f38 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:42:54.624811 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e7d73f85-c16a-4821-afa6-4f3317782f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:42:54.754610 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d73f85-c16a-4821-afa6-4f3317782f38 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:42:54.865597 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:42:54.868589 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e8859-4c3e-40ce-a2eb-10150acb0e30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:42:54.869357 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e8859-4c3e-40ce-a2eb-10150acb0e30 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:42:54.870024 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e8859-4c3e-40ce-a2eb-10150acb0e30 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:42:54.870535 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e8859-4c3e-40ce-a2eb-10150acb0e30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:42:54.872778 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4e3e8859-4c3e-40ce-a2eb-10150acb0e30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:42:55.031968 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e8859-4c3e-40ce-a2eb-10150acb0e30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:42:55.033449 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e8859-4c3e-40ce-a2eb-10150acb0e30 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:42:55.053948 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 665/1314] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:42:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:42:58.027870 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e166da-281e-46f3-8e45-963a94824540 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:42:58.028614 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e166da-281e-46f3-8e45-963a94824540 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:42:58.028614 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e166da-281e-46f3-8e45-963a94824540 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:42:58.029870 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 650/1315] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:42:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:42:58.037351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0146a5f-b9d2-43bd-aa26-2e27c79d5198 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:42:58.038260 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0146a5f-b9d2-43bd-aa26-2e27c79d5198 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:42:58.038260 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0146a5f-b9d2-43bd-aa26-2e27c79d5198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:42:58.131293 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-a0146a5f-b9d2-43bd-aa26-2e27c79d5198 None None] MFA Rules not processed for user `748c27d9ec1c4f29912ec71cb6f8863c`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:42:58.133144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a0146a5f-b9d2-43bd-aa26-2e27c79d5198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:42:58.209802 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:42:58.281094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 666/1316] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:42:58 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:42:58.637744 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e166da-281e-46f3-8e45-963a94824540 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:42:58.639468 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-86e166da-281e-46f3-8e45-963a94824540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:42:58.653447 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:42:58.784371 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e166da-281e-46f3-8e45-963a94824540 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:42:58.875108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689f24f-7c7a-484d-a9fd-b7e5c5bef2cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:42:58.875760 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689f24f-7c7a-484d-a9fd-b7e5c5bef2cf service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:42:58.876042 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689f24f-7c7a-484d-a9fd-b7e5c5bef2cf service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:42:58.876326 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689f24f-7c7a-484d-a9fd-b7e5c5bef2cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:42:58.878085 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a689f24f-7c7a-484d-a9fd-b7e5c5bef2cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:42:58.968924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:42:59.035771 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689f24f-7c7a-484d-a9fd-b7e5c5bef2cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:42:59.037478 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689f24f-7c7a-484d-a9fd-b7e5c5bef2cf service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:42:59.062338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 651/1317] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:42:58 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:43:01.778785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5ffab-db25-45d4-8fd2-713403ae0b03 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:01.779471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5ffab-db25-45d4-8fd2-713403ae0b03 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:01.779737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5ffab-db25-45d4-8fd2-713403ae0b03 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:01.782059 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 667/1318] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:43:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:43:01.791256 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99ca10-fc48-489a-813d-acd955eae8b7 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:01.791976 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99ca10-fc48-489a-813d-acd955eae8b7 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:01.792161 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99ca10-fc48-489a-813d-acd955eae8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:01.902061 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-5f99ca10-fc48-489a-813d-acd955eae8b7 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:43:01.903462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5f99ca10-fc48-489a-813d-acd955eae8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:02.042128 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 652/1319] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:43:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:43:02.394000 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3cb3ce-0c60-4e98-804a-d36e9306aa55 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:02.394508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3cb3ce-0c60-4e98-804a-d36e9306aa55 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:02.395099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3cb3ce-0c60-4e98-804a-d36e9306aa55 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:02.397802 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 668/1320] 10.222.0.71 () {54 vars in 965 bytes} [Fri Aug 26 08:43:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:43:02.408070 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55709805-6d17-4277-a44e-dd45ff587ee1 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:02.408478 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55709805-6d17-4277-a44e-dd45ff587ee1 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:02.408775 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55709805-6d17-4277-a44e-dd45ff587ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:02.449366 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:43:02.525298 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-55709805-6d17-4277-a44e-dd45ff587ee1 None None] MFA Rules not processed for user `4aa8938a504c4c4f8815478e520482d8`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:43:02.527391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-55709805-6d17-4277-a44e-dd45ff587ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:02.641975 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 653/1321] 10.222.0.71 () {58 vars in 1062 bytes} [Fri Aug 26 08:43:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:43:02.652216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3cb3ce-0c60-4e98-804a-d36e9306aa55 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:43:02.653844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-bf3cb3ce-0c60-4e98-804a-d36e9306aa55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:02.818123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3cb3ce-0c60-4e98-804a-d36e9306aa55 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:43:02.942831 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f2f97-4a14-461e-96b3-fc1361ed6584 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:43:02.943845 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f2f97-4a14-461e-96b3-fc1361ed6584 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:02.944361 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f2f97-4a14-461e-96b3-fc1361ed6584 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:02.944857 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f2f97-4a14-461e-96b3-fc1361ed6584 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:02.948042 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-622f2f97-4a14-461e-96b3-fc1361ed6584 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:02.960234 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:43:03.129893 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f2f97-4a14-461e-96b3-fc1361ed6584 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:43:03.131585 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f2f97-4a14-461e-96b3-fc1361ed6584 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:43:03.152810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 669/1322] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:43:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:43:06.517786 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b928259-b0d2-4bb9-ad06-a722b6a9be9e None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:06.518309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b928259-b0d2-4bb9-ad06-a722b6a9be9e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:06.518309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b928259-b0d2-4bb9-ad06-a722b6a9be9e None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:06.519693 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 654/1323] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:43:06.533187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148af8e-2f4b-4e39-86d1-ea8fe8cf9ddb None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:06.535316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148af8e-2f4b-4e39-86d1-ea8fe8cf9ddb None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:06.536308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148af8e-2f4b-4e39-86d1-ea8fe8cf9ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:06.611336 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-d148af8e-2f4b-4e39-86d1-ea8fe8cf9ddb None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:43:06.612757 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d148af8e-2f4b-4e39-86d1-ea8fe8cf9ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:06.683963 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:43:06.743407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 670/1324] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:43:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:43:07.076019 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b928259-b0d2-4bb9-ad06-a722b6a9be9e None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:43:07.078787 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4b928259-b0d2-4bb9-ad06-a722b6a9be9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:07.228811 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b928259-b0d2-4bb9-ad06-a722b6a9be9e None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:43:07.247578 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:43:07.351229 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b98b5-da83-417c-ab65-e8a2ec91244d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:43:07.351922 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b98b5-da83-417c-ab65-e8a2ec91244d service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:07.352397 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b98b5-da83-417c-ab65-e8a2ec91244d service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:07.352812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b98b5-da83-417c-ab65-e8a2ec91244d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:07.355006 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-822b98b5-da83-417c-ab65-e8a2ec91244d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:07.527819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b98b5-da83-417c-ab65-e8a2ec91244d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:43:07.530001 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b98b5-da83-417c-ab65-e8a2ec91244d service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:43:07.551383 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 655/1325] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:43:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:43:09.795930 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993b654-254d-47e6-9cfe-a841153aaa54 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:09.796518 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993b654-254d-47e6-9cfe-a841153aaa54 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:09.796518 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0993b654-254d-47e6-9cfe-a841153aaa54 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:09.798426 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 671/1326] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:43:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:43:09.808018 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5beee6-17ca-4c65-b811-4230436e830e None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:09.808576 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5beee6-17ca-4c65-b811-4230436e830e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:09.808703 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5beee6-17ca-4c65-b811-4230436e830e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:09.867959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:43:09.895038 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-5f5beee6-17ca-4c65-b811-4230436e830e None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:43:09.896329 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5f5beee6-17ca-4c65-b811-4230436e830e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:10.031746 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 656/1327] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:43:10.318543 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993b654-254d-47e6-9cfe-a841153aaa54 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:43:10.320274 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-0993b654-254d-47e6-9cfe-a841153aaa54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:10.470113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0993b654-254d-47e6-9cfe-a841153aaa54 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:43:10.587264 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a153083-61fb-4273-8b33-0fa546b1d210 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:43:10.587834 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a153083-61fb-4273-8b33-0fa546b1d210 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:10.588115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a153083-61fb-4273-8b33-0fa546b1d210 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:10.588384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a153083-61fb-4273-8b33-0fa546b1d210 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:10.590131 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1a153083-61fb-4273-8b33-0fa546b1d210 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:10.613761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:43:10.749210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a153083-61fb-4273-8b33-0fa546b1d210 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:43:10.751261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a153083-61fb-4273-8b33-0fa546b1d210 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:43:10.775032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 672/1328] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:43:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:43:13.011964 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41452531-72ad-49bc-94fc-7e4efa17c42e None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:13.012811 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41452531-72ad-49bc-94fc-7e4efa17c42e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:13.012811 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41452531-72ad-49bc-94fc-7e4efa17c42e None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:13.014912 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 657/1329] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:43:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:43:13.024430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57347260-eb9f-474b-9848-64f41a9ea0d0 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:13.025307 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57347260-eb9f-474b-9848-64f41a9ea0d0 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:13.025307 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57347260-eb9f-474b-9848-64f41a9ea0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:13.146942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-57347260-eb9f-474b-9848-64f41a9ea0d0 None None] MFA Rules not processed for user `748c27d9ec1c4f29912ec71cb6f8863c`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:43:13.148816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-57347260-eb9f-474b-9848-64f41a9ea0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:13.258196 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:43:13.302760 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 673/1330] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:43:13 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:43:13.639562 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41452531-72ad-49bc-94fc-7e4efa17c42e None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:43:13.640993 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-41452531-72ad-49bc-94fc-7e4efa17c42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:13.808212 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41452531-72ad-49bc-94fc-7e4efa17c42e None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:43:13.851566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:43:13.952570 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd40dd77-e372-475f-ad54-670f62db348d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:43:13.953211 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40dd77-e372-475f-ad54-670f62db348d service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:13.953611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40dd77-e372-475f-ad54-670f62db348d service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:13.953889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40dd77-e372-475f-ad54-670f62db348d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:13.955856 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-dd40dd77-e372-475f-ad54-670f62db348d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:14.171243 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd40dd77-e372-475f-ad54-670f62db348d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:43:14.173995 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd40dd77-e372-475f-ad54-670f62db348d service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:43:14.205000 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 658/1331] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:43:13 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:43:16.878217 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a77ea-1350-4ad4-bd91-4dff0296a9e5 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:16.878851 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a77ea-1350-4ad4-bd91-4dff0296a9e5 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:16.879048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a77ea-1350-4ad4-bd91-4dff0296a9e5 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:16.880637 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 674/1332] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:43:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:43:16.887155 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3648612-a2bf-41ec-9e85-948cd5cc370c None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:16.887956 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3648612-a2bf-41ec-9e85-948cd5cc370c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:16.888117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3648612-a2bf-41ec-9e85-948cd5cc370c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:16.977843 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:43:16.980820 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-d3648612-a2bf-41ec-9e85-948cd5cc370c None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:43:16.982614 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d3648612-a2bf-41ec-9e85-948cd5cc370c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:17.101511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 659/1333] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:43:17.385393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a77ea-1350-4ad4-bd91-4dff0296a9e5 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:43:17.387495 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-667a77ea-1350-4ad4-bd91-4dff0296a9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:17.513094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a77ea-1350-4ad4-bd91-4dff0296a9e5 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:43:17.609262 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d005d1c0-811c-440f-981c-cdffcccba42d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:43:17.609894 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005d1c0-811c-440f-981c-cdffcccba42d service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:17.610169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005d1c0-811c-440f-981c-cdffcccba42d service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:17.610436 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005d1c0-811c-440f-981c-cdffcccba42d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:17.612351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d005d1c0-811c-440f-981c-cdffcccba42d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:17.649725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:43:17.752509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d005d1c0-811c-440f-981c-cdffcccba42d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:43:17.753983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d005d1c0-811c-440f-981c-cdffcccba42d service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:43:17.777365 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 675/1334] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:43:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:43:19.923464 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4718b5d8-7a0b-48de-8f5d-a17b3b68df31 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:19.924331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4718b5d8-7a0b-48de-8f5d-a17b3b68df31 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:19.924331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4718b5d8-7a0b-48de-8f5d-a17b3b68df31 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:19.925918 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 660/1335] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:43:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:43:19.932085 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6f95c-f457-456e-8d09-3f0ff4212655 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:19.932669 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6f95c-f457-456e-8d09-3f0ff4212655 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:19.932669 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6f95c-f457-456e-8d09-3f0ff4212655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:20.024293 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-d4d6f95c-f457-456e-8d09-3f0ff4212655 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:43:20.025835 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d4d6f95c-f457-456e-8d09-3f0ff4212655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:20.116527 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:43:20.144768 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 676/1336] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:43:20.443845 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4718b5d8-7a0b-48de-8f5d-a17b3b68df31 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:43:20.445381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4718b5d8-7a0b-48de-8f5d-a17b3b68df31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:20.566115 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4718b5d8-7a0b-48de-8f5d-a17b3b68df31 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:43:20.603751 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:43:20.671093 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1488848-ffa9-4bce-b4b5-38850bc2eb29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:43:20.671956 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1488848-ffa9-4bce-b4b5-38850bc2eb29 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:20.672435 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1488848-ffa9-4bce-b4b5-38850bc2eb29 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:20.672679 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1488848-ffa9-4bce-b4b5-38850bc2eb29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:20.675268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d1488848-ffa9-4bce-b4b5-38850bc2eb29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:20.814709 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1488848-ffa9-4bce-b4b5-38850bc2eb29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:43:20.816527 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1488848-ffa9-4bce-b4b5-38850bc2eb29 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:43:20.840246 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 661/1337] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:43:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:43:23.475078 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7756860-bc5e-422e-bc9d-93056a118929 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:23.475646 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7756860-bc5e-422e-bc9d-93056a118929 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:23.475646 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7756860-bc5e-422e-bc9d-93056a118929 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:23.476877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 677/1338] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:43:23.482944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1151a-8e86-4407-8a30-203ff901dff7 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:23.483481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1151a-8e86-4407-8a30-203ff901dff7 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:23.483481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1151a-8e86-4407-8a30-203ff901dff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:23.582434 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-43b1151a-8e86-4407-8a30-203ff901dff7 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:43:23.584044 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-43b1151a-8e86-4407-8a30-203ff901dff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:23.594474 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:43:23.710720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 662/1339] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:43:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:43:24.010741 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7756860-bc5e-422e-bc9d-93056a118929 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:43:24.012691 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e7756860-bc5e-422e-bc9d-93056a118929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:24.130324 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7756860-bc5e-422e-bc9d-93056a118929 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:43:24.215389 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccfbfd-a04e-4db6-be17-75b04a1a5512 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:43:24.216130 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccfbfd-a04e-4db6-be17-75b04a1a5512 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:24.216454 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccfbfd-a04e-4db6-be17-75b04a1a5512 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:24.216766 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccfbfd-a04e-4db6-be17-75b04a1a5512 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:24.218516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a1ccfbfd-a04e-4db6-be17-75b04a1a5512 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:24.272243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:43:24.388194 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ccfbfd-a04e-4db6-be17-75b04a1a5512 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:43:24.389741 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ccfbfd-a04e-4db6-be17-75b04a1a5512 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:43:24.411954 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 678/1340] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:43:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:43:29.346236 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a632247-4362-43ac-bb94-b85fff8737d5 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:29.347120 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a632247-4362-43ac-bb94-b85fff8737d5 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:29.347214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a632247-4362-43ac-bb94-b85fff8737d5 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:29.349770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 663/1341] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:43:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:43:29.359731 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7345f-8d29-4320-a323-fa6181f32032 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:29.359731 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7345f-8d29-4320-a323-fa6181f32032 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:29.361024 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7345f-8d29-4320-a323-fa6181f32032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:29.471585 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-f4f7345f-8d29-4320-a323-fa6181f32032 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:43:29.473224 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f4f7345f-8d29-4320-a323-fa6181f32032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:29.636483 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:43:29.660242 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 679/1342] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:43:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:43:30.032589 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a632247-4362-43ac-bb94-b85fff8737d5 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:43:30.034867 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1a632247-4362-43ac-bb94-b85fff8737d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:30.206662 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a632247-4362-43ac-bb94-b85fff8737d5 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:43:30.282005 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:43:30.347649 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ad4c6-ee7c-474d-8c79-8dc28dc03068 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:43:30.348812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ad4c6-ee7c-474d-8c79-8dc28dc03068 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:30.349118 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ad4c6-ee7c-474d-8c79-8dc28dc03068 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:30.349590 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ad4c6-ee7c-474d-8c79-8dc28dc03068 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:30.352771 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7d1ad4c6-ee7c-474d-8c79-8dc28dc03068 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:30.560835 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ad4c6-ee7c-474d-8c79-8dc28dc03068 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:43:30.563211 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ad4c6-ee7c-474d-8c79-8dc28dc03068 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:43:30.594057 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 664/1343] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:43:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:43:34.365021 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cc769-7311-48f1-afdd-1d8ad49f10c0 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:34.365726 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cc769-7311-48f1-afdd-1d8ad49f10c0 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:34.365726 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cc769-7311-48f1-afdd-1d8ad49f10c0 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:34.370612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 680/1344] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:43:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:43:34.377195 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de2973-4aef-4606-88d0-1629708e01b4 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:34.378373 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de2973-4aef-4606-88d0-1629708e01b4 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:34.378373 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de2973-4aef-4606-88d0-1629708e01b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:34.497612 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-e4de2973-4aef-4606-88d0-1629708e01b4 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:43:34.499525 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e4de2973-4aef-4606-88d0-1629708e01b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:34.535176 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:43:34.668446 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 665/1345] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:43:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:43:35.068195 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cc769-7311-48f1-afdd-1d8ad49f10c0 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:43:35.070776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-6e9cc769-7311-48f1-afdd-1d8ad49f10c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:35.271832 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cc769-7311-48f1-afdd-1d8ad49f10c0 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:43:35.407530 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149248b6-ac35-44a8-bd14-d54e419beb00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:43:35.408485 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149248b6-ac35-44a8-bd14-d54e419beb00 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:35.408950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149248b6-ac35-44a8-bd14-d54e419beb00 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:35.409389 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149248b6-ac35-44a8-bd14-d54e419beb00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:35.412142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-149248b6-ac35-44a8-bd14-d54e419beb00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:35.490030 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:43:35.625311 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149248b6-ac35-44a8-bd14-d54e419beb00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:43:35.628203 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149248b6-ac35-44a8-bd14-d54e419beb00 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:43:35.662055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 681/1346] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:43:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:43:41.699627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab2dbb-1811-4f92-adb9-a1462081a2b0 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:41.700229 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab2dbb-1811-4f92-adb9-a1462081a2b0 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:41.700229 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab2dbb-1811-4f92-adb9-a1462081a2b0 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:41.702829 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 666/1347] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:43:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:43:41.710769 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36044ae8-881f-4085-844d-86bcebb153fd None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:41.712307 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36044ae8-881f-4085-844d-86bcebb153fd None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:41.712307 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36044ae8-881f-4085-844d-86bcebb153fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:41.825755 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-36044ae8-881f-4085-844d-86bcebb153fd None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:43:41.827608 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-36044ae8-881f-4085-844d-86bcebb153fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:42.019183 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:43:42.029687 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 682/1348] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:43:42.338196 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab2dbb-1811-4f92-adb9-a1462081a2b0 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:43:42.339854 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8fab2dbb-1811-4f92-adb9-a1462081a2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:42.514398 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab2dbb-1811-4f92-adb9-a1462081a2b0 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:43:42.590622 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:43:42.642094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97a49b-1fa2-4ae0-a9bb-1b82a3db7e66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:43:42.643063 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97a49b-1fa2-4ae0-a9bb-1b82a3db7e66 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:42.643520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97a49b-1fa2-4ae0-a9bb-1b82a3db7e66 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:42.644136 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97a49b-1fa2-4ae0-a9bb-1b82a3db7e66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:42.647157 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5e97a49b-1fa2-4ae0-a9bb-1b82a3db7e66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:42.804179 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e97a49b-1fa2-4ae0-a9bb-1b82a3db7e66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:43:42.806089 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e97a49b-1fa2-4ae0-a9bb-1b82a3db7e66 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:43:42.827095 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 667/1349] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:43:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:43:48.699235 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a47ac9-a268-4557-aacf-7522d3acff7f None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:48.700012 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a47ac9-a268-4557-aacf-7522d3acff7f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:48.700012 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a47ac9-a268-4557-aacf-7522d3acff7f None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:48.701700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 683/1350] 10.222.0.71 () {54 vars in 921 bytes} [Fri Aug 26 08:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:43:48.708395 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08317b-bdf3-4d3a-9f28-4030721ead6e None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:48.709237 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08317b-bdf3-4d3a-9f28-4030721ead6e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:48.709237 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08317b-bdf3-4d3a-9f28-4030721ead6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:48.830276 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-9c08317b-bdf3-4d3a-9f28-4030721ead6e None None] MFA Rules not processed for user `0cf5e2b4f5e440a7942adea3d51cec75`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:43:48.831762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9c08317b-bdf3-4d3a-9f28-4030721ead6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:48.864252 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:43:49.033806 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 668/1351] 10.222.0.71 () {58 vars in 1018 bytes} [Fri Aug 26 08:43:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:43:53.742306 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc2bc5-6906-4d85-a720-41ba87074ba0 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:53.743400 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc2bc5-6906-4d85-a720-41ba87074ba0 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:53.743815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc2bc5-6906-4d85-a720-41ba87074ba0 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:53.745655 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 684/1352] 10.222.0.71 () {54 vars in 921 bytes} [Fri Aug 26 08:43:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:43:53.753687 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38036690-5700-42ed-91bf-892bac2f38d4 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:53.754197 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38036690-5700-42ed-91bf-892bac2f38d4 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:53.754446 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38036690-5700-42ed-91bf-892bac2f38d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:53.858479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-38036690-5700-42ed-91bf-892bac2f38d4 None None] MFA Rules not processed for user `0cf5e2b4f5e440a7942adea3d51cec75`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:43:53.860283 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-38036690-5700-42ed-91bf-892bac2f38d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:53.969118 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:43:54.026609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 669/1353] 10.222.0.71 () {58 vars in 1018 bytes} [Fri Aug 26 08:43:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:43:55.313423 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3265c46-fd0e-4082-9fb0-fe5b65db9a18 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:55.314054 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3265c46-fd0e-4082-9fb0-fe5b65db9a18 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:55.314054 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3265c46-fd0e-4082-9fb0-fe5b65db9a18 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:55.315464 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 685/1354] 10.222.0.71 () {54 vars in 921 bytes} [Fri Aug 26 08:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:43:55.320970 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f081f-2c18-4541-8376-7b27da28a3af None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:55.321411 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f081f-2c18-4541-8376-7b27da28a3af None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:55.321591 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f081f-2c18-4541-8376-7b27da28a3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:55.414591 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-336f081f-2c18-4541-8376-7b27da28a3af None None] MFA Rules not processed for user `0cf5e2b4f5e440a7942adea3d51cec75`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:43:55.414591 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-336f081f-2c18-4541-8376-7b27da28a3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:55.536419 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 670/1355] 10.222.0.71 () {58 vars in 1018 bytes} [Fri Aug 26 08:43:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:43:59.024446 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f05980-eb26-42b6-a795-e668fb1deb1e None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:59.025196 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f05980-eb26-42b6-a795-e668fb1deb1e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:59.025196 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f05980-eb26-42b6-a795-e668fb1deb1e None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:59.026737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 686/1356] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:43:59.032598 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6593ebc-aa0f-4a43-addc-5e733f5d3331 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:59.033150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6593ebc-aa0f-4a43-addc-5e733f5d3331 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:59.033260 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6593ebc-aa0f-4a43-addc-5e733f5d3331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:59.074005 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:43:59.116635 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-a6593ebc-aa0f-4a43-addc-5e733f5d3331 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:43:59.117845 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a6593ebc-aa0f-4a43-addc-5e733f5d3331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:59.254983 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 671/1357] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:43:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:43:59.559921 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61257bc1-cb0d-4cf8-b092-5ee24319b006 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:59.560676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61257bc1-cb0d-4cf8-b092-5ee24319b006 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:59.561320 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61257bc1-cb0d-4cf8-b092-5ee24319b006 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:59.564106 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 687/1358] 10.222.0.71 () {54 vars in 962 bytes} [Fri Aug 26 08:43:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:43:59.571836 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af929424-5535-4444-9055-e684d526bfa2 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:43:59.572517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af929424-5535-4444-9055-e684d526bfa2 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:43:59.573401 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af929424-5535-4444-9055-e684d526bfa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:43:59.669926 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-af929424-5535-4444-9055-e684d526bfa2 None None] MFA Rules not processed for user `f16d1768715a41648af3fe6afadd7477`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:43:59.671250 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-af929424-5535-4444-9055-e684d526bfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:59.729152 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:43:59.809091 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 672/1359] 10.222.0.71 () {58 vars in 1059 bytes} [Fri Aug 26 08:43:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:43:59.817945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61257bc1-cb0d-4cf8-b092-5ee24319b006 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:43:59.819518 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-61257bc1-cb0d-4cf8-b092-5ee24319b006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:43:59.976355 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61257bc1-cb0d-4cf8-b092-5ee24319b006 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:44:00.063427 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:00.108484 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de91c0-b08a-4e76-9d2d-7b7f1ee8c75f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:44:00.109085 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de91c0-b08a-4e76-9d2d-7b7f1ee8c75f service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:00.109430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de91c0-b08a-4e76-9d2d-7b7f1ee8c75f service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:00.109737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de91c0-b08a-4e76-9d2d-7b7f1ee8c75f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:00.111743 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d9de91c0-b08a-4e76-9d2d-7b7f1ee8c75f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:00.245035 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de91c0-b08a-4e76-9d2d-7b7f1ee8c75f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:44:00.246187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de91c0-b08a-4e76-9d2d-7b7f1ee8c75f service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:44:00.271449 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 688/1360] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:43:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:44:02.412107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204de795-089d-464e-9d3f-bec33e26d1c2 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:02.412759 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204de795-089d-464e-9d3f-bec33e26d1c2 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:02.412759 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204de795-089d-464e-9d3f-bec33e26d1c2 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:02.414204 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 673/1361] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:44:02.419592 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c9e45-25b3-4a05-bb55-799388939490 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:02.420166 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c9e45-25b3-4a05-bb55-799388939490 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:02.420166 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c9e45-25b3-4a05-bb55-799388939490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:02.497352 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-112c9e45-25b3-4a05-bb55-799388939490 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:44:02.499025 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-112c9e45-25b3-4a05-bb55-799388939490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:02.546135 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:02.650927 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 689/1362] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:44:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:44:02.930258 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204de795-089d-464e-9d3f-bec33e26d1c2 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:44:02.931906 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-204de795-089d-464e-9d3f-bec33e26d1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:03.102752 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204de795-089d-464e-9d3f-bec33e26d1c2 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:44:03.113908 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:03.238536 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb07902-0395-44bc-a302-ba8a8b02f9d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:44:03.239227 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb07902-0395-44bc-a302-ba8a8b02f9d3 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:03.239567 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb07902-0395-44bc-a302-ba8a8b02f9d3 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:03.239902 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb07902-0395-44bc-a302-ba8a8b02f9d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:03.241583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-deb07902-0395-44bc-a302-ba8a8b02f9d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:03.338291 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb07902-0395-44bc-a302-ba8a8b02f9d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:44:03.339928 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb07902-0395-44bc-a302-ba8a8b02f9d3 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:44:03.362188 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 674/1363] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:44:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:44:05.582123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071092a-74d6-4504-af57-ce4e88c75724 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:05.582738 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071092a-74d6-4504-af57-ce4e88c75724 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:05.582738 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071092a-74d6-4504-af57-ce4e88c75724 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:05.584076 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 690/1364] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:44:05.590933 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513a6df-450b-41b4-b162-bf12290da665 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:05.591488 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513a6df-450b-41b4-b162-bf12290da665 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:05.591737 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513a6df-450b-41b4-b162-bf12290da665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:05.609310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:05.682262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-4513a6df-450b-41b4-b162-bf12290da665 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:44:05.683519 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4513a6df-450b-41b4-b162-bf12290da665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:05.815711 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 675/1365] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:44:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:44:06.140274 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c1140-dd54-47ce-a231-3ac7d019132a None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:06.140877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c1140-dd54-47ce-a231-3ac7d019132a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:06.141152 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c1140-dd54-47ce-a231-3ac7d019132a None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:06.143089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 691/1366] 10.222.0.71 () {54 vars in 962 bytes} [Fri Aug 26 08:44:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:44:06.151516 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c7fd1-b1fd-4e9a-969d-d91d80040304 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:06.152193 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c7fd1-b1fd-4e9a-969d-d91d80040304 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:06.152793 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c7fd1-b1fd-4e9a-969d-d91d80040304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:06.263054 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-0d4c7fd1-b1fd-4e9a-969d-d91d80040304 None None] MFA Rules not processed for user `f16d1768715a41648af3fe6afadd7477`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:44:06.265062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0d4c7fd1-b1fd-4e9a-969d-d91d80040304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:06.308100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:06.429877 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 676/1367] 10.222.0.71 () {58 vars in 1059 bytes} [Fri Aug 26 08:44:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:44:06.441214 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c1140-dd54-47ce-a231-3ac7d019132a None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:44:06.444972 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b45c1140-dd54-47ce-a231-3ac7d019132a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:06.628036 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c1140-dd54-47ce-a231-3ac7d019132a None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:44:06.651644 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:06.754991 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41302100-7036-49f3-9eed-a9f7e9e55d2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:44:06.755605 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41302100-7036-49f3-9eed-a9f7e9e55d2c service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:06.755881 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41302100-7036-49f3-9eed-a9f7e9e55d2c service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:06.756147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41302100-7036-49f3-9eed-a9f7e9e55d2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:06.758099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-41302100-7036-49f3-9eed-a9f7e9e55d2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:06.854373 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41302100-7036-49f3-9eed-a9f7e9e55d2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:44:06.856183 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41302100-7036-49f3-9eed-a9f7e9e55d2c service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:44:06.876354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 692/1368] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:44:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:44:07.278714 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e89935-3bcf-47e4-a21d-ed72da2194e8 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:07.279337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e89935-3bcf-47e4-a21d-ed72da2194e8 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:07.279337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e89935-3bcf-47e4-a21d-ed72da2194e8 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:07.280828 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 677/1369] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:44:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:44:07.286403 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74755f2b-e2d9-4c92-be48-5c888bca5a87 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:07.286638 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74755f2b-e2d9-4c92-be48-5c888bca5a87 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:07.286961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74755f2b-e2d9-4c92-be48-5c888bca5a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:07.324368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:07.366931 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-74755f2b-e2d9-4c92-be48-5c888bca5a87 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:44:07.371133 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-74755f2b-e2d9-4c92-be48-5c888bca5a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:07.474338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 693/1370] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:44:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:44:08.059246 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880946a-4578-4202-90f3-188907551671 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:08.059246 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880946a-4578-4202-90f3-188907551671 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:08.059246 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880946a-4578-4202-90f3-188907551671 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:08.062532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 678/1371] 10.222.0.71 () {54 vars in 964 bytes} [Fri Aug 26 08:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:44:08.066207 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba37d4-fc54-4a82-b4ab-4d04291e1b6a None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:08.066992 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba37d4-fc54-4a82-b4ab-4d04291e1b6a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:08.066992 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba37d4-fc54-4a82-b4ab-4d04291e1b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:08.149690 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-adba37d4-fc54-4a82-b4ab-4d04291e1b6a None None] MFA Rules not processed for user `997b4ffc01fc467ebbd1092787f217f6`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:44:08.151240 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-adba37d4-fc54-4a82-b4ab-4d04291e1b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:08.205003 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:08.241454 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 694/1372] 10.222.0.71 () {58 vars in 1061 bytes} [Fri Aug 26 08:44:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:44:08.249720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880946a-4578-4202-90f3-188907551671 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:44:08.251325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6880946a-4578-4202-90f3-188907551671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:08.367777 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880946a-4578-4202-90f3-188907551671 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:44:08.419542 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:08.462394 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0ebe8c-5aa2-4074-806f-c3e7b2bb8757 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:44:08.463040 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ebe8c-5aa2-4074-806f-c3e7b2bb8757 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:08.463327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ebe8c-5aa2-4074-806f-c3e7b2bb8757 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:08.463676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ebe8c-5aa2-4074-806f-c3e7b2bb8757 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:08.465379 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0a0ebe8c-5aa2-4074-806f-c3e7b2bb8757 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:08.601951 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0ebe8c-5aa2-4074-806f-c3e7b2bb8757 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:44:08.603645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0ebe8c-5aa2-4074-806f-c3e7b2bb8757 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:44:08.628843 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 679/1373] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:44:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:44:08.937513 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bcd9c-6748-4586-81db-f939799f9493 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:08.937803 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bcd9c-6748-4586-81db-f939799f9493 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:08.938099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bcd9c-6748-4586-81db-f939799f9493 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:08.939884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 695/1374] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:44:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:44:08.945591 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753ac5f-8379-49e4-b7cb-b88a6c9ebd95 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:08.945880 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753ac5f-8379-49e4-b7cb-b88a6c9ebd95 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:08.946119 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753ac5f-8379-49e4-b7cb-b88a6c9ebd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:09.014994 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-0753ac5f-8379-49e4-b7cb-b88a6c9ebd95 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:44:09.016215 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0753ac5f-8379-49e4-b7cb-b88a6c9ebd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:09.038355 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:09.120083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 680/1375] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:44:09.411979 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8bcd9c-6748-4586-81db-f939799f9493 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:44:09.413529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ed8bcd9c-6748-4586-81db-f939799f9493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:09.567781 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8bcd9c-6748-4586-81db-f939799f9493 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:44:09.623637 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:09.702818 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a9e58-c90a-4920-a536-87f2761f215a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:44:09.703634 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a9e58-c90a-4920-a536-87f2761f215a service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:09.703919 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a9e58-c90a-4920-a536-87f2761f215a service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:09.704191 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a9e58-c90a-4920-a536-87f2761f215a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:09.705953 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8f0a9e58-c90a-4920-a536-87f2761f215a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:09.808651 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0a9e58-c90a-4920-a536-87f2761f215a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:44:09.810278 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0a9e58-c90a-4920-a536-87f2761f215a service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:44:09.829991 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 696/1376] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:44:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:44:11.820761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c5b22-03b0-4003-87c5-220d5224a413 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:11.821346 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c5b22-03b0-4003-87c5-220d5224a413 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:11.821346 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c5b22-03b0-4003-87c5-220d5224a413 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:11.823428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 681/1377] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:44:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:44:11.830034 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af20e2-e147-4c44-966f-d8c060111062 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:11.830576 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af20e2-e147-4c44-966f-d8c060111062 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:11.830686 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af20e2-e147-4c44-966f-d8c060111062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:11.905737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-53af20e2-e147-4c44-966f-d8c060111062 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:44:11.907000 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-53af20e2-e147-4c44-966f-d8c060111062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:11.914352 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:12.015904 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 697/1378] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:44:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:44:12.320584 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c5b22-03b0-4003-87c5-220d5224a413 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:44:12.322088 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-927c5b22-03b0-4003-87c5-220d5224a413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:12.534715 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c5b22-03b0-4003-87c5-220d5224a413 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:44:12.538719 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:12.689983 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef822d-1334-4d00-a39a-3dfcd9231f6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:44:12.690991 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef822d-1334-4d00-a39a-3dfcd9231f6f service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:12.691329 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef822d-1334-4d00-a39a-3dfcd9231f6f service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:12.691843 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef822d-1334-4d00-a39a-3dfcd9231f6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:12.693491 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-88ef822d-1334-4d00-a39a-3dfcd9231f6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:12.798686 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef822d-1334-4d00-a39a-3dfcd9231f6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:44:12.800240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef822d-1334-4d00-a39a-3dfcd9231f6f service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:44:12.821187 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 682/1379] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:44:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:44:14.874636 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9460b64-b716-42e6-8fbc-b208f624d4c5 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:14.875561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9460b64-b716-42e6-8fbc-b208f624d4c5 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:14.875561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9460b64-b716-42e6-8fbc-b208f624d4c5 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:14.876568 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 698/1380] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:44:14.883241 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db24c25-a831-4aa7-8805-04db28fd88da None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:14.883918 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db24c25-a831-4aa7-8805-04db28fd88da None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:14.884429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db24c25-a831-4aa7-8805-04db28fd88da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:14.901709 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:15.000221 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-7db24c25-a831-4aa7-8805-04db28fd88da None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:44:15.001668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7db24c25-a831-4aa7-8805-04db28fd88da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:15.131139 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 683/1381] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:44:15.471849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9460b64-b716-42e6-8fbc-b208f624d4c5 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:44:15.473889 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f9460b64-b716-42e6-8fbc-b208f624d4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:15.671050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:15.693447 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9460b64-b716-42e6-8fbc-b208f624d4c5 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:44:15.868574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6fe7ae-7a15-4f73-9f53-1e147ff2760d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:44:15.869191 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6fe7ae-7a15-4f73-9f53-1e147ff2760d service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:15.869472 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6fe7ae-7a15-4f73-9f53-1e147ff2760d service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:15.869749 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6fe7ae-7a15-4f73-9f53-1e147ff2760d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:15.871394 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-df6fe7ae-7a15-4f73-9f53-1e147ff2760d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:15.980336 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6fe7ae-7a15-4f73-9f53-1e147ff2760d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:44:15.983018 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6fe7ae-7a15-4f73-9f53-1e147ff2760d service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:44:16.009641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:16.013404 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 699/1382] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:44:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:44:18.185643 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee9c1b-be65-459a-966c-c35dca4cd903 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:18.186490 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee9c1b-be65-459a-966c-c35dca4cd903 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:18.186490 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee9c1b-be65-459a-966c-c35dca4cd903 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:18.188027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 684/1383] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:44:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:44:18.195789 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93abc1f-d67c-447e-ba7b-6c513a78b47f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:18.196558 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93abc1f-d67c-447e-ba7b-6c513a78b47f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:18.196558 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93abc1f-d67c-447e-ba7b-6c513a78b47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:18.296490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e93abc1f-d67c-447e-ba7b-6c513a78b47f None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:44:18.298582 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e93abc1f-d67c-447e-ba7b-6c513a78b47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:18.451179 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 700/1384] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:44:18.750350 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee9c1b-be65-459a-966c-c35dca4cd903 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:44:18.751953 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-37ee9c1b-be65-459a-966c-c35dca4cd903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:18.883056 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:18.960974 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee9c1b-be65-459a-966c-c35dca4cd903 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:44:19.134729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99da2dd-81f6-456e-8962-49227d1d0fb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:44:19.135385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99da2dd-81f6-456e-8962-49227d1d0fb6 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:19.135660 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99da2dd-81f6-456e-8962-49227d1d0fb6 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:19.135927 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99da2dd-81f6-456e-8962-49227d1d0fb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:19.137632 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f99da2dd-81f6-456e-8962-49227d1d0fb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:19.233724 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:19.293272 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99da2dd-81f6-456e-8962-49227d1d0fb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:44:19.295657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99da2dd-81f6-456e-8962-49227d1d0fb6 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:44:19.322687 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 685/1385] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:44:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:44:21.668823 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b6deb2-347f-4ea3-b429-d9c0ea68385d None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:21.669482 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b6deb2-347f-4ea3-b429-d9c0ea68385d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:21.669482 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b6deb2-347f-4ea3-b429-d9c0ea68385d None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:21.670765 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 701/1386] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:44:21.676805 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fe8f8-cb5f-4bee-97e2-f284230b4d9a None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:21.677395 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fe8f8-cb5f-4bee-97e2-f284230b4d9a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:21.677395 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fe8f8-cb5f-4bee-97e2-f284230b4d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:21.760016 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-8e9fe8f8-cb5f-4bee-97e2-f284230b4d9a None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:44:21.761295 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8e9fe8f8-cb5f-4bee-97e2-f284230b4d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:21.921979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 686/1387] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:44:22.236053 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b6deb2-347f-4ea3-b429-d9c0ea68385d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:44:22.237814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-67b6deb2-347f-4ea3-b429-d9c0ea68385d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:22.332862 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:22.425337 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b6deb2-347f-4ea3-b429-d9c0ea68385d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:44:22.553193 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4d5e3-f776-4ba5-b337-8515eb0615eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:44:22.553744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4d5e3-f776-4ba5-b337-8515eb0615eb service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:22.554027 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4d5e3-f776-4ba5-b337-8515eb0615eb service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:22.554305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4d5e3-f776-4ba5-b337-8515eb0615eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:22.556226 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-38b4d5e3-f776-4ba5-b337-8515eb0615eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:22.601090 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:22.657042 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4d5e3-f776-4ba5-b337-8515eb0615eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:44:22.658707 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4d5e3-f776-4ba5-b337-8515eb0615eb service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:44:22.680510 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 702/1388] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:44:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:44:23.965575 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db3b79-591d-44fa-a9df-3ac611b736f4 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:23.966200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db3b79-591d-44fa-a9df-3ac611b736f4 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:23.966200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db3b79-591d-44fa-a9df-3ac611b736f4 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:23.967722 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 687/1389] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:44:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:44:23.973222 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ad923-703a-4725-8b26-13cfbfe0d741 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:23.973759 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ad923-703a-4725-8b26-13cfbfe0d741 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:23.973759 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ad923-703a-4725-8b26-13cfbfe0d741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:24.049121 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-3a2ad923-703a-4725-8b26-13cfbfe0d741 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:44:24.050473 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3a2ad923-703a-4725-8b26-13cfbfe0d741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:24.155886 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 703/1390] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:44:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:44:24.457648 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff9b9c-39de-4a88-8362-d2bd161b4423 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:24.458133 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff9b9c-39de-4a88-8362-d2bd161b4423 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:24.458276 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff9b9c-39de-4a88-8362-d2bd161b4423 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:24.459654 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 688/1391] 10.222.0.71 () {54 vars in 967 bytes} [Fri Aug 26 08:44:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:44:24.465878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa391800-13f2-4c1d-85b3-53fae8a4d9d2 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:24.466238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa391800-13f2-4c1d-85b3-53fae8a4d9d2 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:24.466714 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa391800-13f2-4c1d-85b3-53fae8a4d9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:24.487086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:24.564223 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-fa391800-13f2-4c1d-85b3-53fae8a4d9d2 None None] MFA Rules not processed for user `0d178c717a3e49319545e70cb58cbbed`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:44:24.565413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-fa391800-13f2-4c1d-85b3-53fae8a4d9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:24.649963 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 704/1392] 10.222.0.71 () {58 vars in 1064 bytes} [Fri Aug 26 08:44:24 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:44:24.657770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff9b9c-39de-4a88-8362-d2bd161b4423 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:44:24.659314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-64ff9b9c-39de-4a88-8362-d2bd161b4423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:24.691794 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:24.697296 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd341dc-86ed-4b1d-b26d-fc84d43ce86d None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:24.697575 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd341dc-86ed-4b1d-b26d-fc84d43ce86d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:24.697938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd341dc-86ed-4b1d-b26d-fc84d43ce86d None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:24.699411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 705/1393] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:44:24.705628 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45b805-77f9-4943-b8cf-47a703547e4a None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:24.705911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45b805-77f9-4943-b8cf-47a703547e4a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:24.706185 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a45b805-77f9-4943-b8cf-47a703547e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:24.785273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff9b9c-39de-4a88-8362-d2bd161b4423 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:44:24.791612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-2a45b805-77f9-4943-b8cf-47a703547e4a None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:44:24.793088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2a45b805-77f9-4943-b8cf-47a703547e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:24.857887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:24.869372 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee96c0-8276-4980-8819-a772fac0d83f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d178c717a3e49319545e70cb58cbbed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:44:24.869984 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee96c0-8276-4980-8819-a772fac0d83f service glance] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:24.870253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee96c0-8276-4980-8819-a772fac0d83f service glance] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:24.870516 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee96c0-8276-4980-8819-a772fac0d83f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:24.872154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9dee96c0-8276-4980-8819-a772fac0d83f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:24.905343 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 706/1394] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:44:24.965436 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:25.006043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dee96c0-8276-4980-8819-a772fac0d83f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:44:25.007558 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dee96c0-8276-4980-8819-a772fac0d83f service glance] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:44:25.028155 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 689/1395] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:44:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:44:25.218681 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a45b805-77f9-4943-b8cf-47a703547e4a None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:44:25.220610 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2a45b805-77f9-4943-b8cf-47a703547e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:25.371395 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a45b805-77f9-4943-b8cf-47a703547e4a None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:44:25.466345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:25.540480 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360b841-bfed-4145-8111-556c12406cf5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:44:25.540990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360b841-bfed-4145-8111-556c12406cf5 service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:25.541258 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360b841-bfed-4145-8111-556c12406cf5 service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:25.541522 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360b841-bfed-4145-8111-556c12406cf5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:25.543272 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d360b841-bfed-4145-8111-556c12406cf5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:25.678224 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360b841-bfed-4145-8111-556c12406cf5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:44:25.680842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360b841-bfed-4145-8111-556c12406cf5 service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:44:25.703793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 707/1396] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:44:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:44:25.897476 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc8d66-6cd1-498f-afc1-7a891a71f4ae None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:25.897941 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc8d66-6cd1-498f-afc1-7a891a71f4ae None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:25.898399 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc8d66-6cd1-498f-afc1-7a891a71f4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:26.023192 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-a7fc8d66-6cd1-498f-afc1-7a891a71f4ae None None] MFA Rules not processed for user `df07cffd82c74621855c8b8d7bfa989d`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:44:26.025601 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a7fc8d66-6cd1-498f-afc1-7a891a71f4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:26.171414 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 690/1397] 10.222.0.71 () {58 vars in 1009 bytes} [Fri Aug 26 08:44:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:44:26.227302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c07e6-cc16-4920-8217-c2316648bf78 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:26.227963 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c07e6-cc16-4920-8217-c2316648bf78 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:26.227963 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c07e6-cc16-4920-8217-c2316648bf78 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:26.229305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 708/1398] 10.222.0.71 () {54 vars in 943 bytes} [Fri Aug 26 08:44:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:44:26.238857 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7f074-ac3a-4af4-a42a-9d4daba2c0b1 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:26.239202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7f074-ac3a-4af4-a42a-9d4daba2c0b1 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:26.239501 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7f074-ac3a-4af4-a42a-9d4daba2c0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:26.310285 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:26.335939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-57a7f074-ac3a-4af4-a42a-9d4daba2c0b1 None None] MFA Rules not processed for user `1723e961375b45b7b3bd99ddeb69683f`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:44:26.337372 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-57a7f074-ac3a-4af4-a42a-9d4daba2c0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:26.471964 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 691/1399] 10.222.0.71 () {58 vars in 1040 bytes} [Fri Aug 26 08:44:26 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:44:26.482153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c07e6-cc16-4920-8217-c2316648bf78 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:44:26.483921 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-740c07e6-cc16-4920-8217-c2316648bf78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:26.571284 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:26.635372 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c07e6-cc16-4920-8217-c2316648bf78 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:44:26.755609 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4ed79-a9ab-43ad-a0cb-5254dee996be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1723e961375b45b7b3bd99ddeb69683f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:44:26.756278 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4ed79-a9ab-43ad-a0cb-5254dee996be service swift] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:26.756701 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4ed79-a9ab-43ad-a0cb-5254dee996be service swift] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:26.757095 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4ed79-a9ab-43ad-a0cb-5254dee996be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:26.759117 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-33a4ed79-a9ab-43ad-a0cb-5254dee996be service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:26.845433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:26.866127 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a4ed79-a9ab-43ad-a0cb-5254dee996be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:44:26.867804 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a4ed79-a9ab-43ad-a0cb-5254dee996be service swift] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:44:26.876358 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 709/1400] 10.222.0.71 () {60 vars in 1384 bytes} [Fri Aug 26 08:44:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 08:44:27.882701 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08133f61-4060-4c4e-b77f-48b364b53949 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:27.883477 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08133f61-4060-4c4e-b77f-48b364b53949 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:27.883780 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08133f61-4060-4c4e-b77f-48b364b53949 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:27.887454 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 692/1401] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:44:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:44:27.892159 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7fc43-f3a7-4fca-9ba1-a61a3e432b85 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:27.892705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7fc43-f3a7-4fca-9ba1-a61a3e432b85 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:27.892705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7fc43-f3a7-4fca-9ba1-a61a3e432b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:27.996861 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-b2c7fc43-f3a7-4fca-9ba1-a61a3e432b85 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:44:27.998309 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b2c7fc43-f3a7-4fca-9ba1-a61a3e432b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:28.134944 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 710/1402] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:44:28.426228 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08133f61-4060-4c4e-b77f-48b364b53949 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:44:28.427831 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-08133f61-4060-4c4e-b77f-48b364b53949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:28.550338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:28.599375 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08133f61-4060-4c4e-b77f-48b364b53949 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:44:28.747414 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c12b0-6506-4508-8ad9-c2fffedaa626 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:44:28.747926 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c12b0-6506-4508-8ad9-c2fffedaa626 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:28.748202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c12b0-6506-4508-8ad9-c2fffedaa626 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:28.748466 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c12b0-6506-4508-8ad9-c2fffedaa626 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:28.750337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f81c12b0-6506-4508-8ad9-c2fffedaa626 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:28.859519 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:28.888323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c12b0-6506-4508-8ad9-c2fffedaa626 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:44:28.890212 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c12b0-6506-4508-8ad9-c2fffedaa626 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:44:28.915339 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 693/1403] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:44:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:44:31.029885 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f0b06-b9bf-4697-9133-cae731ba2c5b None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:31.030523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f0b06-b9bf-4697-9133-cae731ba2c5b None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:31.030523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f0b06-b9bf-4697-9133-cae731ba2c5b None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:31.036012 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 711/1404] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:44:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:44:31.037984 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321e227-1362-4567-a6ed-84e15133cfbc None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:31.038869 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321e227-1362-4567-a6ed-84e15133cfbc None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:31.039286 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321e227-1362-4567-a6ed-84e15133cfbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:31.121351 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-9321e227-1362-4567-a6ed-84e15133cfbc None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:44:31.122684 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9321e227-1362-4567-a6ed-84e15133cfbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:31.272944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 694/1405] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:44:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:44:31.627499 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f0b06-b9bf-4697-9133-cae731ba2c5b None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:44:31.631182 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4e1f0b06-b9bf-4697-9133-cae731ba2c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:31.734520 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:31.921783 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f0b06-b9bf-4697-9133-cae731ba2c5b None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:44:32.107010 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df407000-e7ba-402b-98d5-ecfeb9b101f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:44:32.107813 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df407000-e7ba-402b-98d5-ecfeb9b101f3 service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:32.108677 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df407000-e7ba-402b-98d5-ecfeb9b101f3 service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:32.109201 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df407000-e7ba-402b-98d5-ecfeb9b101f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:32.111973 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-df407000-e7ba-402b-98d5-ecfeb9b101f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:32.146341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:32.254787 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df407000-e7ba-402b-98d5-ecfeb9b101f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:44:32.256475 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df407000-e7ba-402b-98d5-ecfeb9b101f3 service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:44:32.288357 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 712/1406] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:44:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:44:34.977235 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f6a82-8b2a-417d-aa1d-3e80550422d5 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:34.978449 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f6a82-8b2a-417d-aa1d-3e80550422d5 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:34.978449 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f6a82-8b2a-417d-aa1d-3e80550422d5 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:34.987631 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 695/1407] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:44:34 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:44:34.989181 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6bf28-27c2-409a-a64d-79c8d4a7b711 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:34.989880 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6bf28-27c2-409a-a64d-79c8d4a7b711 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:34.989880 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6bf28-27c2-409a-a64d-79c8d4a7b711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:35.107829 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-37c6bf28-27c2-409a-a64d-79c8d4a7b711 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:44:35.110187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-37c6bf28-27c2-409a-a64d-79c8d4a7b711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:35.253412 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:35.267632 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 713/1408] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:44:35.684682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f6a82-8b2a-417d-aa1d-3e80550422d5 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:44:35.686760 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c47f6a82-8b2a-417d-aa1d-3e80550422d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:35.771479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:35.940474 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f6a82-8b2a-417d-aa1d-3e80550422d5 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:44:36.118120 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ab35a-aa14-48b6-b1ab-2d2795205077 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:44:36.118933 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ab35a-aa14-48b6-b1ab-2d2795205077 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:36.119036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ab35a-aa14-48b6-b1ab-2d2795205077 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:36.119272 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ab35a-aa14-48b6-b1ab-2d2795205077 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:36.120970 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3d3ab35a-aa14-48b6-b1ab-2d2795205077 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:36.129178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:36.241012 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ab35a-aa14-48b6-b1ab-2d2795205077 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:44:36.242851 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ab35a-aa14-48b6-b1ab-2d2795205077 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:44:36.269414 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 696/1409] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:44:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:44:45.263925 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:46.266657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=42 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:52.520639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0a4a5-e06f-4b99-8e46-ed8cb0c6276c None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:52.521205 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0a4a5-e06f-4b99-8e46-ed8cb0c6276c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:52.521205 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0a4a5-e06f-4b99-8e46-ed8cb0c6276c None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:52.522509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 714/1410] 10.222.0.71 () {54 vars in 918 bytes} [Fri Aug 26 08:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:44:52.528812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2da1ca-262c-4182-9739-39c4abdee89e None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:52.529316 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2da1ca-262c-4182-9739-39c4abdee89e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:52.529469 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2da1ca-262c-4182-9739-39c4abdee89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:52.637721 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-1b2da1ca-262c-4182-9739-39c4abdee89e None None] MFA Rules not processed for user `0cf5e2b4f5e440a7942adea3d51cec75`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:44:52.639651 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1b2da1ca-262c-4182-9739-39c4abdee89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:52.777695 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 697/1411] 10.222.0.71 () {58 vars in 1015 bytes} [Fri Aug 26 08:44:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:44:52.795760 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db126b-f089-4d80-a80a-5e5238e20227 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:52.796041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db126b-f089-4d80-a80a-5e5238e20227 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:52.796319 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db126b-f089-4d80-a80a-5e5238e20227 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:52.797719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 715/1412] 10.222.0.71 () {54 vars in 918 bytes} [Fri Aug 26 08:44:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:44:52.804159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa9826-7335-496c-a01c-37c7dcf81112 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:52.804453 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa9826-7335-496c-a01c-37c7dcf81112 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:52.804726 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa9826-7335-496c-a01c-37c7dcf81112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:52.911280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:52.915050 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c1aa9826-7335-496c-a01c-37c7dcf81112 None None] MFA Rules not processed for user `f16d1768715a41648af3fe6afadd7477`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:44:52.916514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c1aa9826-7335-496c-a01c-37c7dcf81112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:53.106459 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 698/1413] 10.222.0.71 () {58 vars in 1015 bytes} [Fri Aug 26 08:44:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:44:55.802378 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a6949-50fc-4e08-8e38-f8c37752493c None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:55.803159 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a6949-50fc-4e08-8e38-f8c37752493c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:55.803159 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a6949-50fc-4e08-8e38-f8c37752493c None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:55.804421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 716/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 08:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:44:55.811345 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6378f-d156-4917-ad89-1e9a72b5368f None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:55.811950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6378f-d156-4917-ad89-1e9a72b5368f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:55.811950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6378f-d156-4917-ad89-1e9a72b5368f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:55.926306 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-66f6378f-d156-4917-ad89-1e9a72b5368f None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:44:55.927898 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-66f6378f-d156-4917-ad89-1e9a72b5368f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:56.018732 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:56.084999 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 699/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 08:44:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:44:56.420242 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a6949-50fc-4e08-8e38-f8c37752493c None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:44:56.422004 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4e0a6949-50fc-4e08-8e38-f8c37752493c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:56.620158 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a6949-50fc-4e08-8e38-f8c37752493c None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:44:56.721328 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:56.786430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce86c1-4345-4adc-a915-7946df891da8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:44:56.787654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce86c1-4345-4adc-a915-7946df891da8 service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:56.788189 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce86c1-4345-4adc-a915-7946df891da8 service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:56.788871 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce86c1-4345-4adc-a915-7946df891da8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:56.792372 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-9dce86c1-4345-4adc-a915-7946df891da8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:56.927371 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce86c1-4345-4adc-a915-7946df891da8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:44:56.929039 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce86c1-4345-4adc-a915-7946df891da8 service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:44:56.950677 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 717/1416] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:44:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:44:57.904684 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c702a-0daf-439f-b515-6be80630c684 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:57.905281 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c702a-0daf-439f-b515-6be80630c684 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:57.905674 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c702a-0daf-439f-b515-6be80630c684 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:57.908261 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 700/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 08:44:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:44:57.915584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17349a-6d6d-409e-bc66-7642af197700 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:57.916018 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17349a-6d6d-409e-bc66-7642af197700 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:57.916377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17349a-6d6d-409e-bc66-7642af197700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:58.014429 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-2d17349a-6d6d-409e-bc66-7642af197700 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:44:58.015693 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2d17349a-6d6d-409e-bc66-7642af197700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:58.046345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:58.146534 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 718/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 08:44:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:44:58.307757 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c702a-0daf-439f-b515-6be80630c684 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:44:58.309376 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-686c702a-0daf-439f-b515-6be80630c684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:58.515931 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c702a-0daf-439f-b515-6be80630c684 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:44:58.564432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:58.665696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfe19c-1363-41a4-a6f5-a4e3245ca5d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:44:58.666710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfe19c-1363-41a4-a6f5-a4e3245ca5d0 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:58.667467 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfe19c-1363-41a4-a6f5-a4e3245ca5d0 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:58.668100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfe19c-1363-41a4-a6f5-a4e3245ca5d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:58.673468 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-39cfe19c-1363-41a4-a6f5-a4e3245ca5d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:44:58.792894 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cfe19c-1363-41a4-a6f5-a4e3245ca5d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:44:58.794634 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cfe19c-1363-41a4-a6f5-a4e3245ca5d0 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:44:58.834252 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 701/1419] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:44:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:44:59.882308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edd2e7-b4fb-4b72-9668-11a1c57456c0 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:59.882933 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edd2e7-b4fb-4b72-9668-11a1c57456c0 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:59.882933 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edd2e7-b4fb-4b72-9668-11a1c57456c0 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:59.884718 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 719/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 08:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:44:59.893368 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ae0d7-8e99-4c62-b855-f93613013092 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:44:59.893907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ae0d7-8e99-4c62-b855-f93613013092 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:44:59.893907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ae0d7-8e99-4c62-b855-f93613013092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:44:59.954113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:44:59.979585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-a86ae0d7-8e99-4c62-b855-f93613013092 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:44:59.980817 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a86ae0d7-8e99-4c62-b855-f93613013092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:00.093826 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 702/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 08:44:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:45:00.245876 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edd2e7-b4fb-4b72-9668-11a1c57456c0 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:45:00.247673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a5edd2e7-b4fb-4b72-9668-11a1c57456c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:00.447721 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edd2e7-b4fb-4b72-9668-11a1c57456c0 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:45:00.460549 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:00.570540 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244247e8-1e63-43c6-92bf-19dc5837c226 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:45:00.571647 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244247e8-1e63-43c6-92bf-19dc5837c226 service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:00.571916 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244247e8-1e63-43c6-92bf-19dc5837c226 service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:00.572170 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244247e8-1e63-43c6-92bf-19dc5837c226 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:00.573825 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-244247e8-1e63-43c6-92bf-19dc5837c226 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:00.677498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244247e8-1e63-43c6-92bf-19dc5837c226 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:45:00.679431 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244247e8-1e63-43c6-92bf-19dc5837c226 service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:45:00.698831 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 720/1422] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:45:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:45:01.658800 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b50de-f4f0-4d41-b722-dccf5cd38e27 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:01.659392 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b50de-f4f0-4d41-b722-dccf5cd38e27 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:01.659392 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b50de-f4f0-4d41-b722-dccf5cd38e27 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:01.660873 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 703/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 08:45:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:45:01.668062 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872e669-79e8-4717-8ebd-8c9a5f44e202 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:01.668062 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872e669-79e8-4717-8ebd-8c9a5f44e202 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:01.668655 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872e669-79e8-4717-8ebd-8c9a5f44e202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:01.694038 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:01.770443 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e872e669-79e8-4717-8ebd-8c9a5f44e202 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:45:01.771800 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e872e669-79e8-4717-8ebd-8c9a5f44e202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:01.879346 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 721/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 08:45:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:45:02.066503 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b50de-f4f0-4d41-b722-dccf5cd38e27 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:45:02.068149 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c42b50de-f4f0-4d41-b722-dccf5cd38e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:02.200128 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:02.239693 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b50de-f4f0-4d41-b722-dccf5cd38e27 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:45:02.379670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a552f254-ec3b-4aa6-a98f-389d56b8b065 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:45:02.380282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552f254-ec3b-4aa6-a98f-389d56b8b065 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:02.380557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552f254-ec3b-4aa6-a98f-389d56b8b065 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:02.380826 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552f254-ec3b-4aa6-a98f-389d56b8b065 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:02.382424 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a552f254-ec3b-4aa6-a98f-389d56b8b065 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:02.474820 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:02.490131 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a552f254-ec3b-4aa6-a98f-389d56b8b065 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:45:02.493377 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a552f254-ec3b-4aa6-a98f-389d56b8b065 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:45:02.512178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 704/1425] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:45:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:45:03.419336 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0e01e-3f2f-4301-9be0-d82a3768fccd None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:03.419977 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0e01e-3f2f-4301-9be0-d82a3768fccd None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:03.419977 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0e01e-3f2f-4301-9be0-d82a3768fccd None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:03.421403 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 722/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 08:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:45:03.427065 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484bf60-b123-4901-8e70-c6cf676a9858 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:03.427543 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484bf60-b123-4901-8e70-c6cf676a9858 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:03.427636 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484bf60-b123-4901-8e70-c6cf676a9858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:03.515739 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-8484bf60-b123-4901-8e70-c6cf676a9858 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:45:03.516980 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8484bf60-b123-4901-8e70-c6cf676a9858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:03.621710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 705/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 08:45:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:45:03.775078 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0e01e-3f2f-4301-9be0-d82a3768fccd None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:45:03.776612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a0b0e01e-3f2f-4301-9be0-d82a3768fccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:03.891791 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:03.976734 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0e01e-3f2f-4301-9be0-d82a3768fccd None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:45:04.097626 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f505e8-904f-4bfb-b150-48c8979bf086 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:45:04.098158 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f505e8-904f-4bfb-b150-48c8979bf086 service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:04.098440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f505e8-904f-4bfb-b150-48c8979bf086 service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:04.098705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f505e8-904f-4bfb-b150-48c8979bf086 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:04.100458 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-88f505e8-904f-4bfb-b150-48c8979bf086 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:04.158143 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:04.209178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f505e8-904f-4bfb-b150-48c8979bf086 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:45:04.210844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f505e8-904f-4bfb-b150-48c8979bf086 service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:45:04.229324 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 723/1428] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:45:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:45:05.047226 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84f338-670d-4dfd-9d25-e57afffacd0d None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:05.048069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84f338-670d-4dfd-9d25-e57afffacd0d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:05.048069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84f338-670d-4dfd-9d25-e57afffacd0d None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:05.049190 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 706/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 08:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:45:05.055312 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77532267-42fe-4708-9784-355363991ba2 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:05.055859 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77532267-42fe-4708-9784-355363991ba2 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:05.055859 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77532267-42fe-4708-9784-355363991ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:05.154809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-77532267-42fe-4708-9784-355363991ba2 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:45:05.156072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-77532267-42fe-4708-9784-355363991ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:05.271082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 724/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 08:45:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:45:05.423809 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84f338-670d-4dfd-9d25-e57afffacd0d None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:45:05.425352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ee84f338-670d-4dfd-9d25-e57afffacd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:05.479331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:05.601112 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84f338-670d-4dfd-9d25-e57afffacd0d None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:45:05.731063 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25261b45-dfc9-4828-985b-5f1fe612d7e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:45:05.731695 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25261b45-dfc9-4828-985b-5f1fe612d7e7 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:05.731969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25261b45-dfc9-4828-985b-5f1fe612d7e7 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:05.732248 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25261b45-dfc9-4828-985b-5f1fe612d7e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:05.733852 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-25261b45-dfc9-4828-985b-5f1fe612d7e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:05.741107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:05.836602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25261b45-dfc9-4828-985b-5f1fe612d7e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:45:05.838259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25261b45-dfc9-4828-985b-5f1fe612d7e7 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:45:05.860413 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 707/1431] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:45:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:45:06.693083 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c351f-0435-43b3-a635-c7990a8cff5d None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:06.693928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c351f-0435-43b3-a635-c7990a8cff5d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:06.693928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c351f-0435-43b3-a635-c7990a8cff5d None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:06.695263 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 725/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 08:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:45:06.700638 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b02e73-3bfb-44b3-bf13-0361e7a7cbdb None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:06.701116 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b02e73-3bfb-44b3-bf13-0361e7a7cbdb None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:06.701234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b02e73-3bfb-44b3-bf13-0361e7a7cbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:06.781456 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-f5b02e73-3bfb-44b3-bf13-0361e7a7cbdb None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:45:06.782663 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f5b02e73-3bfb-44b3-bf13-0361e7a7cbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:06.867648 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:06.903704 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 708/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 08:45:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:45:07.088970 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c351f-0435-43b3-a635-c7990a8cff5d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:45:07.090545 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-7e2c351f-0435-43b3-a635-c7990a8cff5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:07.114751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:07.254733 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c351f-0435-43b3-a635-c7990a8cff5d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:45:07.366001 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:07.414124 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fdf57e-9241-4e00-8c0d-d6f1bf880d2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:45:07.414688 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdf57e-9241-4e00-8c0d-d6f1bf880d2a service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:07.415169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdf57e-9241-4e00-8c0d-d6f1bf880d2a service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:07.415867 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdf57e-9241-4e00-8c0d-d6f1bf880d2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:07.418232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d0fdf57e-9241-4e00-8c0d-d6f1bf880d2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:07.544792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fdf57e-9241-4e00-8c0d-d6f1bf880d2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:45:07.547944 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fdf57e-9241-4e00-8c0d-d6f1bf880d2a service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:45:07.570916 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 726/1434] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:45:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:45:08.422366 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a52c2-daae-4fb4-9948-524b352bc098 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:08.422991 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a52c2-daae-4fb4-9948-524b352bc098 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:08.423327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a52c2-daae-4fb4-9948-524b352bc098 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:08.425585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 709/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 08:45:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:45:08.436011 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32981832-e875-47ab-a0c6-aff126f3ca29 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:08.436695 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32981832-e875-47ab-a0c6-aff126f3ca29 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:08.436695 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32981832-e875-47ab-a0c6-aff126f3ca29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:08.526617 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-32981832-e875-47ab-a0c6-aff126f3ca29 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:45:08.527911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-32981832-e875-47ab-a0c6-aff126f3ca29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:08.581933 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:08.654183 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 727/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 08:45:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:45:08.810568 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a52c2-daae-4fb4-9948-524b352bc098 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:45:08.812343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6c7a52c2-daae-4fb4-9948-524b352bc098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:08.985362 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a52c2-daae-4fb4-9948-524b352bc098 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:45:09.044925 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:09.112096 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2bc23d-d75f-4d50-bed2-7638a66c3dec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:45:09.112728 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bc23d-d75f-4d50-bed2-7638a66c3dec service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:09.113006 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bc23d-d75f-4d50-bed2-7638a66c3dec service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:09.113273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bc23d-d75f-4d50-bed2-7638a66c3dec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:09.114958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-be2bc23d-d75f-4d50-bed2-7638a66c3dec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:09.208469 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2bc23d-d75f-4d50-bed2-7638a66c3dec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:45:09.210125 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2bc23d-d75f-4d50-bed2-7638a66c3dec service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:45:09.232745 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 710/1437] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:45:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:45:10.050769 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d69f7f-cc4b-46ee-b188-4f30ec1ea218 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:10.051713 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d69f7f-cc4b-46ee-b188-4f30ec1ea218 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:10.051713 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d69f7f-cc4b-46ee-b188-4f30ec1ea218 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:10.052623 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 728/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 08:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:45:10.058055 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf527448-2275-4ee0-878e-c129d33dde4e None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:10.058516 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf527448-2275-4ee0-878e-c129d33dde4e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:10.058612 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf527448-2275-4ee0-878e-c129d33dde4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:10.141852 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-bf527448-2275-4ee0-878e-c129d33dde4e None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:45:10.143092 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-bf527448-2275-4ee0-878e-c129d33dde4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:10.145157 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:10.274732 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 711/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 08:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:45:10.436530 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d69f7f-cc4b-46ee-b188-4f30ec1ea218 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:45:10.438178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-17d69f7f-cc4b-46ee-b188-4f30ec1ea218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:10.627440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d69f7f-cc4b-46ee-b188-4f30ec1ea218 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:45:10.661730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:10.774031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece317d-32b5-49bc-a86b-d27d4af25660 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:45:10.774648 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece317d-32b5-49bc-a86b-d27d4af25660 service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:10.774952 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece317d-32b5-49bc-a86b-d27d4af25660 service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:10.775221 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece317d-32b5-49bc-a86b-d27d4af25660 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:10.776878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-5ece317d-32b5-49bc-a86b-d27d4af25660 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:10.928236 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece317d-32b5-49bc-a86b-d27d4af25660 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:45:10.930031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece317d-32b5-49bc-a86b-d27d4af25660 service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:45:10.950990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 729/1440] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:45:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:45:12.818230 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa467b3-f9ce-4691-aef8-00b879d6bd26 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:12.818833 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa467b3-f9ce-4691-aef8-00b879d6bd26 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:12.818833 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa467b3-f9ce-4691-aef8-00b879d6bd26 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:12.820489 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 712/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 08:45:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:45:12.827044 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bf1d1-c7d5-42d8-8c9f-c9f30fb3f03c None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:12.827612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bf1d1-c7d5-42d8-8c9f-c9f30fb3f03c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:12.827612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bf1d1-c7d5-42d8-8c9f-c9f30fb3f03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:12.874638 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:12.903777 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-571bf1d1-c7d5-42d8-8c9f-c9f30fb3f03c None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:45:12.905061 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-571bf1d1-c7d5-42d8-8c9f-c9f30fb3f03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:13.045692 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 730/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 08:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:45:13.327994 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa467b3-f9ce-4691-aef8-00b879d6bd26 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:45:13.329778 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6fa467b3-f9ce-4691-aef8-00b879d6bd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:13.462901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa467b3-f9ce-4691-aef8-00b879d6bd26 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:45:13.508747 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:13.563946 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76579ff-3e37-4541-b9e2-a51fb07d0648 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:45:13.564879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76579ff-3e37-4541-b9e2-a51fb07d0648 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:13.565212 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76579ff-3e37-4541-b9e2-a51fb07d0648 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:13.565489 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76579ff-3e37-4541-b9e2-a51fb07d0648 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:13.567180 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e76579ff-3e37-4541-b9e2-a51fb07d0648 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:13.715382 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76579ff-3e37-4541-b9e2-a51fb07d0648 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:45:13.716858 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76579ff-3e37-4541-b9e2-a51fb07d0648 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:45:13.737437 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 713/1443] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:45:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:45:15.938562 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185cbcb-9db8-4d34-8f41-614a43d8d690 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:15.939419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185cbcb-9db8-4d34-8f41-614a43d8d690 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:15.939419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185cbcb-9db8-4d34-8f41-614a43d8d690 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:15.940515 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 731/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 08:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:45:15.946583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e24c70-f6ed-4ee3-967e-fec205ed857b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:15.947409 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e24c70-f6ed-4ee3-967e-fec205ed857b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:15.947409 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e24c70-f6ed-4ee3-967e-fec205ed857b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:16.028460 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c8e24c70-f6ed-4ee3-967e-fec205ed857b None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:45:16.029698 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c8e24c70-f6ed-4ee3-967e-fec205ed857b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:16.043401 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:16.163388 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 714/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 08:45:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:45:16.448420 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185cbcb-9db8-4d34-8f41-614a43d8d690 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:45:16.450020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2185cbcb-9db8-4d34-8f41-614a43d8d690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:16.579501 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185cbcb-9db8-4d34-8f41-614a43d8d690 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:45:16.595183 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:16.665478 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181ee46-811e-469f-abba-f1fe757d5a67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:45:16.666213 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181ee46-811e-469f-abba-f1fe757d5a67 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:16.666375 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181ee46-811e-469f-abba-f1fe757d5a67 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:16.666634 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181ee46-811e-469f-abba-f1fe757d5a67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:16.668304 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-5181ee46-811e-469f-abba-f1fe757d5a67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:16.790360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181ee46-811e-469f-abba-f1fe757d5a67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:45:16.791947 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181ee46-811e-469f-abba-f1fe757d5a67 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:45:16.816967 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 732/1446] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:45:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:45:19.140737 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd327586-44b7-4e1e-8267-7a72d5771a99 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:19.141602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd327586-44b7-4e1e-8267-7a72d5771a99 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:19.141602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd327586-44b7-4e1e-8267-7a72d5771a99 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:19.143393 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 715/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 08:45:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:45:19.151662 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42830f-a660-497f-882c-e32823b46db8 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:19.152447 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42830f-a660-497f-882c-e32823b46db8 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:19.152630 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42830f-a660-497f-882c-e32823b46db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:19.205130 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:19.227497 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ac42830f-a660-497f-882c-e32823b46db8 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:45:19.228778 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ac42830f-a660-497f-882c-e32823b46db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:19.346838 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 733/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 08:45:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:45:19.670781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd327586-44b7-4e1e-8267-7a72d5771a99 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:45:19.672388 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-fd327586-44b7-4e1e-8267-7a72d5771a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:19.824149 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd327586-44b7-4e1e-8267-7a72d5771a99 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:45:19.883265 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:19.933302 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8431a53-9e87-4f07-aa36-c9f483d8a23f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:45:19.933817 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8431a53-9e87-4f07-aa36-c9f483d8a23f service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:19.934095 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8431a53-9e87-4f07-aa36-c9f483d8a23f service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:19.934368 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8431a53-9e87-4f07-aa36-c9f483d8a23f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:19.936063 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c8431a53-9e87-4f07-aa36-c9f483d8a23f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:20.066489 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8431a53-9e87-4f07-aa36-c9f483d8a23f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:45:20.068005 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8431a53-9e87-4f07-aa36-c9f483d8a23f service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:45:20.093533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 716/1449] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:45:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:45:22.218040 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f7d66-5e4e-4088-ac69-b44ddce30f98 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:22.218674 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f7d66-5e4e-4088-ac69-b44ddce30f98 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:22.218674 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f7d66-5e4e-4088-ac69-b44ddce30f98 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:22.220071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 734/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 08:45:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:45:22.227059 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb652e8c-847c-4a7e-81bd-5d567c802e45 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:22.227616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb652e8c-847c-4a7e-81bd-5d567c802e45 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:22.227754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb652e8c-847c-4a7e-81bd-5d567c802e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:22.318222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-cb652e8c-847c-4a7e-81bd-5d567c802e45 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:45:22.319519 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-cb652e8c-847c-4a7e-81bd-5d567c802e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:22.345646 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:22.448882 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 717/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 08:45:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:45:22.750517 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f7d66-5e4e-4088-ac69-b44ddce30f98 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:45:22.752212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c93f7d66-5e4e-4088-ac69-b44ddce30f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:22.879668 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f7d66-5e4e-4088-ac69-b44ddce30f98 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:45:22.901480 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:22.975902 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905832a-3d55-485b-8317-16ac4e5af7c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:45:22.976541 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905832a-3d55-485b-8317-16ac4e5af7c9 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:22.976839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905832a-3d55-485b-8317-16ac4e5af7c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:22.977109 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905832a-3d55-485b-8317-16ac4e5af7c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:22.978764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f905832a-3d55-485b-8317-16ac4e5af7c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:23.113796 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905832a-3d55-485b-8317-16ac4e5af7c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:45:23.115391 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905832a-3d55-485b-8317-16ac4e5af7c9 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:45:23.137319 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 735/1452] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:45:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:45:25.311075 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441ed76-7aed-4123-81a0-2ea228784ad1 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:25.311923 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441ed76-7aed-4123-81a0-2ea228784ad1 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:25.311923 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441ed76-7aed-4123-81a0-2ea228784ad1 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:25.312978 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 718/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 08:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:45:25.319613 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011b35b-734f-4809-81e9-a7715294ee4a None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:25.320461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011b35b-734f-4809-81e9-a7715294ee4a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:25.320461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011b35b-734f-4809-81e9-a7715294ee4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:25.412006 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:25.415528 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-4011b35b-734f-4809-81e9-a7715294ee4a None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:45:25.417829 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4011b35b-734f-4809-81e9-a7715294ee4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:25.524407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 736/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 08:45:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:45:25.920983 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441ed76-7aed-4123-81a0-2ea228784ad1 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:45:25.923739 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e441ed76-7aed-4123-81a0-2ea228784ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:26.048507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441ed76-7aed-4123-81a0-2ea228784ad1 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:45:26.111876 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:26.142602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13924aed-0fb5-42a2-b096-eae7cf01f013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:45:26.143248 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13924aed-0fb5-42a2-b096-eae7cf01f013 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:26.143508 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13924aed-0fb5-42a2-b096-eae7cf01f013 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:26.143782 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13924aed-0fb5-42a2-b096-eae7cf01f013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:26.145434 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-13924aed-0fb5-42a2-b096-eae7cf01f013 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:26.269547 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13924aed-0fb5-42a2-b096-eae7cf01f013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:45:26.271459 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13924aed-0fb5-42a2-b096-eae7cf01f013 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:45:26.303889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 719/1455] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:45:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:45:28.420719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f4b3c-28e3-4171-a6d5-d4de25e887f3 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:28.421333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f4b3c-28e3-4171-a6d5-d4de25e887f3 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:28.421333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f4b3c-28e3-4171-a6d5-d4de25e887f3 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:28.422768 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 737/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 08:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:45:28.430335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfea134-2ecc-4ecb-a2a3-9a806c4f9497 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:28.430957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfea134-2ecc-4ecb-a2a3-9a806c4f9497 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:28.430957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfea134-2ecc-4ecb-a2a3-9a806c4f9497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:28.517676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-6cfea134-2ecc-4ecb-a2a3-9a806c4f9497 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:45:28.518954 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6cfea134-2ecc-4ecb-a2a3-9a806c4f9497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:28.552206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:28.640161 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 720/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 08:45:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:45:28.967160 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f4b3c-28e3-4171-a6d5-d4de25e887f3 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:45:28.968799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8e4f4b3c-28e3-4171-a6d5-d4de25e887f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:29.125932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f4b3c-28e3-4171-a6d5-d4de25e887f3 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:45:29.168026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:29.245251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45956d5-c089-496d-82a5-afb2406f1b4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:45:29.246213 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45956d5-c089-496d-82a5-afb2406f1b4e service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:29.246665 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45956d5-c089-496d-82a5-afb2406f1b4e service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:29.247189 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45956d5-c089-496d-82a5-afb2406f1b4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:29.250615 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e45956d5-c089-496d-82a5-afb2406f1b4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:29.412961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45956d5-c089-496d-82a5-afb2406f1b4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:45:29.414652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45956d5-c089-496d-82a5-afb2406f1b4e service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:45:29.444776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 738/1458] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:45:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:45:31.597095 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8337b28-9288-4a5c-a758-be26554f6877 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:31.597681 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8337b28-9288-4a5c-a758-be26554f6877 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:31.597681 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8337b28-9288-4a5c-a758-be26554f6877 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:31.599170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 721/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 08:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:45:31.605750 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f69e4-7601-4dc9-871b-38b80345c0e0 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:31.606473 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f69e4-7601-4dc9-871b-38b80345c0e0 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:31.606473 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f69e4-7601-4dc9-871b-38b80345c0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:31.714168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-948f69e4-7601-4dc9-871b-38b80345c0e0 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:45:31.715459 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-948f69e4-7601-4dc9-871b-38b80345c0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:31.724983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:31.831960 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 739/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 08:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:45:32.209366 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8337b28-9288-4a5c-a758-be26554f6877 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:45:32.211443 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b8337b28-9288-4a5c-a758-be26554f6877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:32.361358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8337b28-9288-4a5c-a758-be26554f6877 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:45:32.476132 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:32.514409 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a4f27-335e-4f2b-b012-6862a8454028 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:45:32.515480 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a4f27-335e-4f2b-b012-6862a8454028 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:32.515965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a4f27-335e-4f2b-b012-6862a8454028 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:32.516440 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a4f27-335e-4f2b-b012-6862a8454028 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:32.519692 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-507a4f27-335e-4f2b-b012-6862a8454028 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:32.714285 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a4f27-335e-4f2b-b012-6862a8454028 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:45:32.716858 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a4f27-335e-4f2b-b012-6862a8454028 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:45:32.747002 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 722/1461] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:45:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:45:35.153484 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9936fc9-d6d3-41a4-92ea-f163fdd433e6 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:35.154088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9936fc9-d6d3-41a4-92ea-f163fdd433e6 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:35.154214 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9936fc9-d6d3-41a4-92ea-f163fdd433e6 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:35.156177 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 740/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 08:45:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:45:35.165479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c1d29-5bd8-44a9-93f8-5921b1c7741c None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:35.166115 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c1d29-5bd8-44a9-93f8-5921b1c7741c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:35.166284 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c1d29-5bd8-44a9-93f8-5921b1c7741c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:35.265704 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-909c1d29-5bd8-44a9-93f8-5921b1c7741c None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:45:35.267341 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-909c1d29-5bd8-44a9-93f8-5921b1c7741c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:35.325237 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:35.412948 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 723/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 08:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:45:35.702732 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9936fc9-d6d3-41a4-92ea-f163fdd433e6 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:45:35.704743 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b9936fc9-d6d3-41a4-92ea-f163fdd433e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:35.824682 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9936fc9-d6d3-41a4-92ea-f163fdd433e6 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:45:35.874317 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:35.930221 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e31738-4736-4c63-97a9-53a7199f9d92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:45:35.930864 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e31738-4736-4c63-97a9-53a7199f9d92 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:35.931174 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e31738-4736-4c63-97a9-53a7199f9d92 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:35.931467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e31738-4736-4c63-97a9-53a7199f9d92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:35.933199 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c4e31738-4736-4c63-97a9-53a7199f9d92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:36.080201 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e31738-4736-4c63-97a9-53a7199f9d92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:45:36.082131 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e31738-4736-4c63-97a9-53a7199f9d92 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:45:36.115908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 741/1464] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:45:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:45:38.407638 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c51e99-475c-49f6-8510-6a9a5489c0a4 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:38.408764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c51e99-475c-49f6-8510-6a9a5489c0a4 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:38.408764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c51e99-475c-49f6-8510-6a9a5489c0a4 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:38.410789 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 724/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 08:45:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:45:38.420547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf18c8-0f6f-4683-b70b-7e733e920d6a None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:38.421536 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf18c8-0f6f-4683-b70b-7e733e920d6a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:38.421862 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf18c8-0f6f-4683-b70b-7e733e920d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:38.513750 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-fcaf18c8-0f6f-4683-b70b-7e733e920d6a None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:45:38.515920 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-fcaf18c8-0f6f-4683-b70b-7e733e920d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:38.541457 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:38.675199 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 742/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 08:45:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:45:38.984858 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c51e99-475c-49f6-8510-6a9a5489c0a4 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:45:38.986393 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e5c51e99-475c-49f6-8510-6a9a5489c0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:39.135798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c51e99-475c-49f6-8510-6a9a5489c0a4 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:45:39.226664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:39.237374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6a31e-7c38-48b4-b213-12425373bac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:45:39.238204 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6a31e-7c38-48b4-b213-12425373bac1 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:39.238653 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6a31e-7c38-48b4-b213-12425373bac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:39.238869 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6a31e-7c38-48b4-b213-12425373bac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:39.241264 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8bb6a31e-7c38-48b4-b213-12425373bac1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:39.418141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6a31e-7c38-48b4-b213-12425373bac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:45:39.420709 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6a31e-7c38-48b4-b213-12425373bac1 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:45:39.456383 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 725/1467] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:45:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:45:41.842198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2755af-844f-4baf-a352-a8bc769ca895 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:41.843381 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2755af-844f-4baf-a352-a8bc769ca895 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:41.843381 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2755af-844f-4baf-a352-a8bc769ca895 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:41.844641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 743/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 08:45:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:45:41.852856 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db7c52-345f-4a21-bce7-1ce840297486 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:41.853685 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db7c52-345f-4a21-bce7-1ce840297486 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:41.854477 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db7c52-345f-4a21-bce7-1ce840297486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:41.956848 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-03db7c52-345f-4a21-bce7-1ce840297486 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:45:41.958701 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-03db7c52-345f-4a21-bce7-1ce840297486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:42.031178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:42.102988 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 726/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 08:45:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:45:42.438562 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2755af-844f-4baf-a352-a8bc769ca895 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:45:42.441420 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ce2755af-844f-4baf-a352-a8bc769ca895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:42.624216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2755af-844f-4baf-a352-a8bc769ca895 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:45:42.689871 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:42.724383 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92447ef-baa1-455f-b2b9-46d1b4f47506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:45:42.724980 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92447ef-baa1-455f-b2b9-46d1b4f47506 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:45:42.725349 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92447ef-baa1-455f-b2b9-46d1b4f47506 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:45:42.725626 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92447ef-baa1-455f-b2b9-46d1b4f47506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:45:42.727373 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f92447ef-baa1-455f-b2b9-46d1b4f47506 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:45:42.886209 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92447ef-baa1-455f-b2b9-46d1b4f47506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:45:42.888220 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92447ef-baa1-455f-b2b9-46d1b4f47506 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:45:42.916265 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 744/1470] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:45:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:45:52.100938 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=22 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:45:52.912095 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=66 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:46:03.415378 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d10b7-58ef-4e3d-878d-aee3850074db None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:46:03.416012 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d10b7-58ef-4e3d-878d-aee3850074db None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:46:03.416012 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d10b7-58ef-4e3d-878d-aee3850074db None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:46:03.417455 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 727/1471] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:46:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:46:03.423880 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd43cc-425d-481a-85f9-e77fb3ad780b None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:46:03.424412 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd43cc-425d-481a-85f9-e77fb3ad780b None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:46:03.424512 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd43cc-425d-481a-85f9-e77fb3ad780b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:46:03.510778 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-b5bd43cc-425d-481a-85f9-e77fb3ad780b None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:46:03.512063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b5bd43cc-425d-481a-85f9-e77fb3ad780b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:46:03.614667 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 745/1472] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:46:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:46:03.913831 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a70b67-3df6-4c50-a088-62f169e32f95 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:46:03.914308 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a70b67-3df6-4c50-a088-62f169e32f95 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:46:03.914752 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a70b67-3df6-4c50-a088-62f169e32f95 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:46:03.917146 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 728/1473] 10.222.0.71 () {54 vars in 967 bytes} [Fri Aug 26 08:46:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:46:03.928259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4505b97-c6f7-41a8-9b89-1237b412d834 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:46:03.928537 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4505b97-c6f7-41a8-9b89-1237b412d834 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:46:03.928803 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4505b97-c6f7-41a8-9b89-1237b412d834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:46:04.014137 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-c4505b97-c6f7-41a8-9b89-1237b412d834 None None] MFA Rules not processed for user `0d178c717a3e49319545e70cb58cbbed`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:46:04.015407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c4505b97-c6f7-41a8-9b89-1237b412d834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:46:04.036504 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:46:04.094259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 746/1474] 10.222.0.71 () {58 vars in 1064 bytes} [Fri Aug 26 08:46:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:46:04.101995 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a70b67-3df6-4c50-a088-62f169e32f95 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:46:04.103583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-25a70b67-3df6-4c50-a088-62f169e32f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:46:04.214931 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a70b67-3df6-4c50-a088-62f169e32f95 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:46:04.293382 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b1ec8-614e-4747-ae88-a882740e8e93 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d178c717a3e49319545e70cb58cbbed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:46:04.294005 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b1ec8-614e-4747-ae88-a882740e8e93 service glance] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:46:04.294285 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b1ec8-614e-4747-ae88-a882740e8e93 service glance] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:46:04.294557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b1ec8-614e-4747-ae88-a882740e8e93 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:46:04.296188 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-089b1ec8-614e-4747-ae88-a882740e8e93 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:46:04.357580 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:46:04.418425 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b1ec8-614e-4747-ae88-a882740e8e93 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:46:04.419952 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b1ec8-614e-4747-ae88-a882740e8e93 service glance] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:46:04.439311 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 729/1475] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:46:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:46:06.509140 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848f98d-bfb8-42a3-a7cc-b0fed19e538f None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:46:06.509793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848f98d-bfb8-42a3-a7cc-b0fed19e538f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:46:06.509793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848f98d-bfb8-42a3-a7cc-b0fed19e538f None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:46:06.511106 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 747/1476] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:46:06.517499 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6b677-a6e0-42e6-98c5-163aae4af0f4 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:46:06.518080 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6b677-a6e0-42e6-98c5-163aae4af0f4 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:46:06.518080 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6b677-a6e0-42e6-98c5-163aae4af0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:46:06.586165 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-1cd6b677-a6e0-42e6-98c5-163aae4af0f4 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:46:06.587442 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1cd6b677-a6e0-42e6-98c5-163aae4af0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:46:06.691732 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 730/1477] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:46:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:46:06.969678 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848f98d-bfb8-42a3-a7cc-b0fed19e538f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:46:06.971267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d848f98d-bfb8-42a3-a7cc-b0fed19e538f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:46:07.104097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d848f98d-bfb8-42a3-a7cc-b0fed19e538f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:46:07.157735 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:46:07.210093 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f57192-c5d7-46ac-ac5a-d59571666cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:46:07.210721 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f57192-c5d7-46ac-ac5a-d59571666cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:46:07.211020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f57192-c5d7-46ac-ac5a-d59571666cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:46:07.211287 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f57192-c5d7-46ac-ac5a-d59571666cf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:46:07.211940 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f57192-c5d7-46ac-ac5a-d59571666cf6 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:46:07.213744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: WARNING py.warnings [None req-11f57192-c5d7-46ac-ac5a-d59571666cf6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 08:46:07.213744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: warnings.warn(msg) Aug 26 08:46:07.213744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: Aug 26 08:46:07.214919 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f57192-c5d7-46ac-ac5a-d59571666cf6 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:46:07.235785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 748/1478] 10.222.0.71 () {56 vars in 1108 bytes} [Fri Aug 26 08:46:06 2022] GET /identity/v3/projects => generated 2553 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:46:09.211069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31366be1-ba49-4ff2-a9ad-2ff8541a10ca None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:46:09.211688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31366be1-ba49-4ff2-a9ad-2ff8541a10ca None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:46:09.211688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31366be1-ba49-4ff2-a9ad-2ff8541a10ca None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:46:09.212965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 731/1479] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:46:09.220037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd57e5-18ee-4526-b11c-27e76e258eaf None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:46:09.220668 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd57e5-18ee-4526-b11c-27e76e258eaf None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:46:09.220668 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd57e5-18ee-4526-b11c-27e76e258eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:46:09.422225 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-40fd57e5-18ee-4526-b11c-27e76e258eaf None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:46:09.423561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-40fd57e5-18ee-4526-b11c-27e76e258eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:46:09.529787 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 749/1480] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:46:09.832437 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31366be1-ba49-4ff2-a9ad-2ff8541a10ca None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:46:09.834166 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-31366be1-ba49-4ff2-a9ad-2ff8541a10ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:46:09.845831 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:46:10.008333 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31366be1-ba49-4ff2-a9ad-2ff8541a10ca None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:46:10.242506 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:46:10.284414 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd9e44-33d7-43b3-af3d-4f9f8569ebe1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:46:10.285067 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd9e44-33d7-43b3-af3d-4f9f8569ebe1 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:46:10.285331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd9e44-33d7-43b3-af3d-4f9f8569ebe1 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:46:10.285592 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd9e44-33d7-43b3-af3d-4f9f8569ebe1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:46:10.287214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c9bd9e44-33d7-43b3-af3d-4f9f8569ebe1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:46:10.389554 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd9e44-33d7-43b3-af3d-4f9f8569ebe1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:46:10.391228 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd9e44-33d7-43b3-af3d-4f9f8569ebe1 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:46:10.410462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 732/1481] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:46:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:46:12.588667 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5d692-5d7e-427a-a306-e27551a778a2 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:46:12.589272 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5d692-5d7e-427a-a306-e27551a778a2 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:46:12.589272 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5d692-5d7e-427a-a306-e27551a778a2 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:46:12.590601 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 750/1482] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:46:12.596705 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac659c6-b801-4cc6-a8f7-99b7e5208912 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:46:12.597193 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac659c6-b801-4cc6-a8f7-99b7e5208912 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:46:12.597296 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac659c6-b801-4cc6-a8f7-99b7e5208912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:46:12.667136 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-0ac659c6-b801-4cc6-a8f7-99b7e5208912 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:46:12.668361 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0ac659c6-b801-4cc6-a8f7-99b7e5208912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:46:12.716215 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:46:12.770764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 733/1483] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:46:13.060620 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5d692-5d7e-427a-a306-e27551a778a2 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:46:13.062220 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-dcb5d692-5d7e-427a-a306-e27551a778a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:46:13.090984 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:46:13.183051 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb5d692-5d7e-427a-a306-e27551a778a2 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:46:13.262250 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eded99-0b17-4afa-8b53-3559b2231018 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:46:13.262862 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eded99-0b17-4afa-8b53-3559b2231018 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:46:13.263164 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eded99-0b17-4afa-8b53-3559b2231018 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:46:13.263427 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eded99-0b17-4afa-8b53-3559b2231018 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:46:13.265543 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d3eded99-0b17-4afa-8b53-3559b2231018 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:46:13.348127 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:46:13.393734 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eded99-0b17-4afa-8b53-3559b2231018 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:46:13.395255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eded99-0b17-4afa-8b53-3559b2231018 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:46:13.420316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 751/1484] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:46:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:46:15.406039 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc4628-af18-42b3-a39f-9093e1f515da None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:46:15.406620 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc4628-af18-42b3-a39f-9093e1f515da None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:46:15.406752 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc4628-af18-42b3-a39f-9093e1f515da None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:46:15.408110 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 734/1485] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:46:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:46:15.417178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6a47a-4ba5-445f-abba-d2e79e6f4023 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:46:15.417712 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6a47a-4ba5-445f-abba-d2e79e6f4023 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:46:15.417712 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6a47a-4ba5-445f-abba-d2e79e6f4023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:46:15.496390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-69e6a47a-4ba5-445f-abba-d2e79e6f4023 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:46:15.497631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-69e6a47a-4ba5-445f-abba-d2e79e6f4023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:46:15.611069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 752/1486] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:46:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:46:15.904371 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc4628-af18-42b3-a39f-9093e1f515da None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:46:15.906165 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6fbc4628-af18-42b3-a39f-9093e1f515da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:46:16.034507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc4628-af18-42b3-a39f-9093e1f515da None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:46:16.117003 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:46:16.118714 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3625c-1572-4fed-b32b-e5049f65fb3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:46:16.119401 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3625c-1572-4fed-b32b-e5049f65fb3f service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:46:16.119665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3625c-1572-4fed-b32b-e5049f65fb3f service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:46:16.119948 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3625c-1572-4fed-b32b-e5049f65fb3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:46:16.122555 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-85b3625c-1572-4fed-b32b-e5049f65fb3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:46:16.243970 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b3625c-1572-4fed-b32b-e5049f65fb3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:46:16.245440 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b3625c-1572-4fed-b32b-e5049f65fb3f service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:46:16.268257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 735/1487] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:46:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:46:18.369882 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f1262-3adc-49a5-909c-6ba9cb92e55e None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:46:18.370474 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f1262-3adc-49a5-909c-6ba9cb92e55e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:46:18.370474 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f1262-3adc-49a5-909c-6ba9cb92e55e None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:46:18.372208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 753/1488] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:46:18.379122 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687544e2-d6da-4981-85f1-96453e6da936 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:46:18.379572 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687544e2-d6da-4981-85f1-96453e6da936 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:46:18.379665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687544e2-d6da-4981-85f1-96453e6da936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:46:18.450076 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-687544e2-d6da-4981-85f1-96453e6da936 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:46:18.451360 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-687544e2-d6da-4981-85f1-96453e6da936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:46:18.505991 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:46:18.557495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 736/1489] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:46:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:46:18.857048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f1262-3adc-49a5-909c-6ba9cb92e55e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:46:18.859237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-057f1262-3adc-49a5-909c-6ba9cb92e55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:46:18.900525 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:46:18.986582 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f1262-3adc-49a5-909c-6ba9cb92e55e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:46:19.072376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23a4e3-2762-466a-98a3-479777a72305 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:46:19.073151 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23a4e3-2762-466a-98a3-479777a72305 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:46:19.073426 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23a4e3-2762-466a-98a3-479777a72305 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:46:19.073694 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23a4e3-2762-466a-98a3-479777a72305 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:46:19.075532 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1c23a4e3-2762-466a-98a3-479777a72305 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:46:19.164825 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:46:19.196496 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23a4e3-2762-466a-98a3-479777a72305 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:46:19.198123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23a4e3-2762-466a-98a3-479777a72305 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:46:19.223617 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 754/1490] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:46:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:46:21.483669 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c8350-344d-42e8-8ff9-575868dc61f4 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:46:21.484594 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c8350-344d-42e8-8ff9-575868dc61f4 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:46:21.484594 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c8350-344d-42e8-8ff9-575868dc61f4 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:46:21.486292 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 737/1491] 10.222.0.71 () {54 vars in 927 bytes} [Fri Aug 26 08:46:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:46:21.492505 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540ab16-b411-4338-a6da-9277c494f154 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:46:21.493153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540ab16-b411-4338-a6da-9277c494f154 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:46:21.493153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540ab16-b411-4338-a6da-9277c494f154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:46:21.592824 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-7540ab16-b411-4338-a6da-9277c494f154 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:46:21.594265 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-7540ab16-b411-4338-a6da-9277c494f154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:46:21.714569 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 755/1492] 10.222.0.71 () {58 vars in 1024 bytes} [Fri Aug 26 08:46:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:46:22.001068 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c8350-344d-42e8-8ff9-575868dc61f4 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:46:22.002645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4f4c8350-344d-42e8-8ff9-575868dc61f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:46:22.134947 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c8350-344d-42e8-8ff9-575868dc61f4 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:46:22.239788 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5763080-beb8-43f6-9613-5357c30b6afe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:46:22.240149 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5763080-beb8-43f6-9613-5357c30b6afe service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:46:22.240349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5763080-beb8-43f6-9613-5357c30b6afe service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:46:22.240616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5763080-beb8-43f6-9613-5357c30b6afe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:46:22.242330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a5763080-beb8-43f6-9613-5357c30b6afe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:46:22.253653 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:46:22.381098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5763080-beb8-43f6-9613-5357c30b6afe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:46:22.382594 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5763080-beb8-43f6-9613-5357c30b6afe service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:46:22.408389 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 738/1493] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:46:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:46:31.712546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=82 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:46:32.404504 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=50 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:47:44.006607 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af092be5-44aa-4b88-ba71-9f59558705b6 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:47:44.007305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af092be5-44aa-4b88-ba71-9f59558705b6 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:47:44.007305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af092be5-44aa-4b88-ba71-9f59558705b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:47:44.090267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-af092be5-44aa-4b88-ba71-9f59558705b6 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:47:44.091571 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-af092be5-44aa-4b88-ba71-9f59558705b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:47:44.208772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 756/1494] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:47:44.219962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389341c-3d18-44d3-9cf3-d1def09b567b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:47:44.220822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389341c-3d18-44d3-9cf3-d1def09b567b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:47:44.220822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389341c-3d18-44d3-9cf3-d1def09b567b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:47:44.308157 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-4389341c-3d18-44d3-9cf3-d1def09b567b None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:47:44.309413 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4389341c-3d18-44d3-9cf3-d1def09b567b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:47:44.424225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 739/1495] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:47:44.430035 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af092be5-44aa-4b88-ba71-9f59558705b6 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:47:44.431492 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-af092be5-44aa-4b88-ba71-9f59558705b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:47:44.444773 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af092be5-44aa-4b88-ba71-9f59558705b6 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:47:44.520538 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:47:44.558226 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac182ecd-eb6e-4d33-a127-e8a2374e4548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:47:44.558914 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac182ecd-eb6e-4d33-a127-e8a2374e4548 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:47:44.559147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac182ecd-eb6e-4d33-a127-e8a2374e4548 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:47:44.559435 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac182ecd-eb6e-4d33-a127-e8a2374e4548 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:47:44.560086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac182ecd-eb6e-4d33-a127-e8a2374e4548 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:47:44.560378 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac182ecd-eb6e-4d33-a127-e8a2374e4548 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:47:44.561416 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: WARNING py.warnings [None req-ac182ecd-eb6e-4d33-a127-e8a2374e4548 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 08:47:44.561416 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: warnings.warn(msg) Aug 26 08:47:44.561416 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: Aug 26 08:47:44.562296 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac182ecd-eb6e-4d33-a127-e8a2374e4548 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:47:44.574983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 757/1496] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:47:44.587178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389341c-3d18-44d3-9cf3-d1def09b567b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:47:44.599071 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389341c-3d18-44d3-9cf3-d1def09b567b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:47:44.674269 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:47:44.717226 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3c89a-3ff2-4f07-a67e-741ce9b5e833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:47:44.717855 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3c89a-3ff2-4f07-a67e-741ce9b5e833 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:47:44.718127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3c89a-3ff2-4f07-a67e-741ce9b5e833 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:47:44.718393 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3c89a-3ff2-4f07-a67e-741ce9b5e833 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:47:44.719091 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c3c89a-3ff2-4f07-a67e-741ce9b5e833 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:47:44.720939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: WARNING py.warnings [None req-f4c3c89a-3ff2-4f07-a67e-741ce9b5e833 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 08:47:44.720939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: warnings.warn(msg) Aug 26 08:47:44.720939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: Aug 26 08:47:44.722155 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c3c89a-3ff2-4f07-a67e-741ce9b5e833 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:47:44.795829 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 740/1497] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:47:44 2022] POST /identity/v3/projects => generated 379 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:47:44.800404 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac182ecd-eb6e-4d33-a127-e8a2374e4548 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:47:44.802351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ac182ecd-eb6e-4d33-a127-e8a2374e4548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:47:44.845149 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac182ecd-eb6e-4d33-a127-e8a2374e4548 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:47:44.979381 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28164fb2-568d-4e10-8f05-e931d33dde80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:47:44.980401 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28164fb2-568d-4e10-8f05-e931d33dde80 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:47:44.980867 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28164fb2-568d-4e10-8f05-e931d33dde80 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:47:44.981286 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28164fb2-568d-4e10-8f05-e931d33dde80 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:47:44.982165 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28164fb2-568d-4e10-8f05-e931d33dde80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:47:44.984089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: WARNING py.warnings [None req-28164fb2-568d-4e10-8f05-e931d33dde80 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 08:47:44.984089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: warnings.warn(msg) Aug 26 08:47:44.984089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: Aug 26 08:47:44.985604 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28164fb2-568d-4e10-8f05-e931d33dde80 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:47:45.048974 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 758/1498] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:47:44 2022] POST /identity/v3/users => generated 290 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:47:45.054551 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3c89a-3ff2-4f07-a67e-741ce9b5e833 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:47:45.056248 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f4c3c89a-3ff2-4f07-a67e-741ce9b5e833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:47:45.153309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3c89a-3ff2-4f07-a67e-741ce9b5e833 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:47:45.282079 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:47:45.306062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171700ad-d4cd-4b13-9ae8-aa89d949f8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:47:45.306772 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171700ad-d4cd-4b13-9ae8-aa89d949f8d8 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:47:45.307154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171700ad-d4cd-4b13-9ae8-aa89d949f8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:47:45.307469 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171700ad-d4cd-4b13-9ae8-aa89d949f8d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:47:45.308529 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171700ad-d4cd-4b13-9ae8-aa89d949f8d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:47:45.309690 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: WARNING py.warnings [None req-171700ad-d4cd-4b13-9ae8-aa89d949f8d8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 08:47:45.309690 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: warnings.warn(msg) Aug 26 08:47:45.309690 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: Aug 26 08:47:45.310751 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171700ad-d4cd-4b13-9ae8-aa89d949f8d8 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:47:45.326054 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 741/1499] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 08:47:45.330633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28164fb2-568d-4e10-8f05-e931d33dde80 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:47:45.332384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-28164fb2-568d-4e10-8f05-e931d33dde80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:47:45.344876 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28164fb2-568d-4e10-8f05-e931d33dde80 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:47:45.378555 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:47:45.471480 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa62e13-2097-40e5-b71b-78d06bdbc7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:47:45.472198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa62e13-2097-40e5-b71b-78d06bdbc7f2 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:47:45.472463 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa62e13-2097-40e5-b71b-78d06bdbc7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:47:45.472743 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa62e13-2097-40e5-b71b-78d06bdbc7f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:47:45.473408 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa62e13-2097-40e5-b71b-78d06bdbc7f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:47:45.474459 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: WARNING py.warnings [None req-7fa62e13-2097-40e5-b71b-78d06bdbc7f2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 08:47:45.474459 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: warnings.warn(msg) Aug 26 08:47:45.474459 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: Aug 26 08:47:45.475453 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa62e13-2097-40e5-b71b-78d06bdbc7f2 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:47:45.490196 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 759/1500] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:47:45.496984 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171700ad-d4cd-4b13-9ae8-aa89d949f8d8 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:47:45.498579 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-171700ad-d4cd-4b13-9ae8-aa89d949f8d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:47:45.539631 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171700ad-d4cd-4b13-9ae8-aa89d949f8d8 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:47:45.665772 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e8128-0827-40e9-a2b2-4e80769b9431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:47:45.666475 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e8128-0827-40e9-a2b2-4e80769b9431 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:47:45.666749 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e8128-0827-40e9-a2b2-4e80769b9431 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:47:45.667044 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e8128-0827-40e9-a2b2-4e80769b9431 admin admin] PATH_INFO: `/v3/projects/313a1631eda04cc59bc71a84839ddbcd/users/7573f8b809f24451bcc246a5516eef0c/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:47:45.693235 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6e8128-0827-40e9-a2b2-4e80769b9431 admin admin] RBAC: Authorizing `identity:create_grant(project_id=313a1631eda04cc59bc71a84839ddbcd, user_id=7573f8b809f24451bcc246a5516eef0c, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:47:45.694295 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: WARNING py.warnings [None req-7b6e8128-0827-40e9-a2b2-4e80769b9431 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 08:47:45.694295 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: warnings.warn(msg) Aug 26 08:47:45.694295 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: Aug 26 08:47:45.696442 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6e8128-0827-40e9-a2b2-4e80769b9431 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:47:45.728246 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 742/1501] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:47:45 2022] PUT /identity/v3/projects/313a1631eda04cc59bc71a84839ddbcd/users/7573f8b809f24451bcc246a5516eef0c/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:47:45.741452 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eac498-b9d2-4885-9269-d90b4bc206e1 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:47:45.742100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eac498-b9d2-4885-9269-d90b4bc206e1 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:47:45.742290 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eac498-b9d2-4885-9269-d90b4bc206e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:47:45.810996 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-f6eac498-b9d2-4885-9269-d90b4bc206e1 None None] MFA Rules not processed for user `7573f8b809f24451bcc246a5516eef0c`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:47:45.812262 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f6eac498-b9d2-4885-9269-d90b4bc206e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:47:45.931380 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 760/1502] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:47:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:47:45.949921 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721384b-2224-4240-ac4c-98b616db002b None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:47:45.950264 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721384b-2224-4240-ac4c-98b616db002b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:47:45.950493 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721384b-2224-4240-ac4c-98b616db002b None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:47:45.951944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 743/1503] 10.222.0.71 () {54 vars in 964 bytes} [Fri Aug 26 08:47:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:47:45.958880 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b2433-d7fe-4e63-b3e7-97408f12dc6f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:47:45.959197 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b2433-d7fe-4e63-b3e7-97408f12dc6f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:47:45.959477 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b2433-d7fe-4e63-b3e7-97408f12dc6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:47:45.995822 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:47:46.058657 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-442b2433-d7fe-4e63-b3e7-97408f12dc6f None None] MFA Rules not processed for user `997b4ffc01fc467ebbd1092787f217f6`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:47:46.060466 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-442b2433-d7fe-4e63-b3e7-97408f12dc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:47:46.165501 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 761/1504] 10.222.0.71 () {58 vars in 1061 bytes} [Fri Aug 26 08:47:45 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:47:46.177949 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6721384b-2224-4240-ac4c-98b616db002b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:47:46.179419 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6721384b-2224-4240-ac4c-98b616db002b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:47:46.254961 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:47:46.336908 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6721384b-2224-4240-ac4c-98b616db002b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:47:46.430813 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970a9e8-e7c9-4b09-9ff0-2b3e25fd4544 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:47:46.431372 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970a9e8-e7c9-4b09-9ff0-2b3e25fd4544 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:47:46.431652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970a9e8-e7c9-4b09-9ff0-2b3e25fd4544 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:47:46.431924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970a9e8-e7c9-4b09-9ff0-2b3e25fd4544 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:47:46.433612 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9970a9e8-e7c9-4b09-9ff0-2b3e25fd4544 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:47:46.558456 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9970a9e8-e7c9-4b09-9ff0-2b3e25fd4544 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:47:46.560950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9970a9e8-e7c9-4b09-9ff0-2b3e25fd4544 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:47:46.568186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:47:46.598590 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 744/1505] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:47:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:47:46.654695 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af07476-3229-4a25-82e3-d5b73073cd51 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:47:46.655196 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af07476-3229-4a25-82e3-d5b73073cd51 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:47:46.655670 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af07476-3229-4a25-82e3-d5b73073cd51 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:47:46.657987 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 762/1506] 10.222.0.71 () {52 vars in 830 bytes} [Fri Aug 26 08:47:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:47:46.663321 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970a9e8-e7c9-4b09-9ff0-2b3e25fd4544 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:47:46.664562 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9970a9e8-e7c9-4b09-9ff0-2b3e25fd4544 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:47:46.809089 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970a9e8-e7c9-4b09-9ff0-2b3e25fd4544 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:47:46.933411 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2409363-dd1e-483d-9799-c907ee67e808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:47:46.934074 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2409363-dd1e-483d-9799-c907ee67e808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:47:46.934354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2409363-dd1e-483d-9799-c907ee67e808 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:47:46.934621 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2409363-dd1e-483d-9799-c907ee67e808 admin admin] PATH_INFO: `/v3/users/7573f8b809f24451bcc246a5516eef0c` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:47:46.948751 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2409363-dd1e-483d-9799-c907ee67e808 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7573f8b809f24451bcc246a5516eef0c)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:47:46.949825 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: WARNING py.warnings [None req-a2409363-dd1e-483d-9799-c907ee67e808 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 08:47:46.949825 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: warnings.warn(msg) Aug 26 08:47:46.949825 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: Aug 26 08:47:46.951109 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2409363-dd1e-483d-9799-c907ee67e808 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:47:46.962602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:47:47.028231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a2409363-dd1e-483d-9799-c907ee67e808 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7573f8b809f24451bcc246a5516eef0c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:47:47.039904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a2409363-dd1e-483d-9799-c907ee67e808 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7573f8b809f24451bcc246a5516eef0c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:47:47.057779 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a2409363-dd1e-483d-9799-c907ee67e808 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7573f8b809f24451bcc246a5516eef0c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:47:47.058269 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a2409363-dd1e-483d-9799-c907ee67e808 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7573f8b809f24451bcc246a5516eef0c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:47:47.058943 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a2409363-dd1e-483d-9799-c907ee67e808 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7573f8b809f24451bcc246a5516eef0c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:47:47.089475 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 745/1507] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:47:46 2022] DELETE /identity/v3/users/7573f8b809f24451bcc246a5516eef0c => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:47:47.094301 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933dfdb-148c-4b80-bd1b-cda218aa96e3 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:47:47.094855 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933dfdb-148c-4b80-bd1b-cda218aa96e3 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:47:47.094992 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933dfdb-148c-4b80-bd1b-cda218aa96e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:47:47.161402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-2933dfdb-148c-4b80-bd1b-cda218aa96e3 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:47:47.162796 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2933dfdb-148c-4b80-bd1b-cda218aa96e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:47:47.243998 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:47:47.260155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 763/1508] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:47:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:47:47.274315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2409363-dd1e-483d-9799-c907ee67e808 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:47:47.276334 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a2409363-dd1e-483d-9799-c907ee67e808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:47:47.406255 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2409363-dd1e-483d-9799-c907ee67e808 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:47:47.481174 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:47:47.506624 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682fe26-bfde-4498-9bd3-6d7246051298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:47:47.507344 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682fe26-bfde-4498-9bd3-6d7246051298 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:47:47.507684 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682fe26-bfde-4498-9bd3-6d7246051298 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:47:47.508003 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682fe26-bfde-4498-9bd3-6d7246051298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:47:47.509975 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d682fe26-bfde-4498-9bd3-6d7246051298 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:47:47.662354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d682fe26-bfde-4498-9bd3-6d7246051298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:47:47.664441 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d682fe26-bfde-4498-9bd3-6d7246051298 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:47:47.684944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 746/1509] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:47:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:47:48.114169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933dfdb-148c-4b80-bd1b-cda218aa96e3 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:47:48.117202 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2933dfdb-148c-4b80-bd1b-cda218aa96e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:47:48.138093 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2933dfdb-148c-4b80-bd1b-cda218aa96e3 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:47:48.327727 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25369e31-fec1-4eb6-8971-7ef40e025c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:47:48.328814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25369e31-fec1-4eb6-8971-7ef40e025c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:47:48.329297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25369e31-fec1-4eb6-8971-7ef40e025c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:47:48.329769 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25369e31-fec1-4eb6-8971-7ef40e025c82 admin admin] PATH_INFO: `/v3/projects/313a1631eda04cc59bc71a84839ddbcd` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:47:48.350408 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25369e31-fec1-4eb6-8971-7ef40e025c82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=313a1631eda04cc59bc71a84839ddbcd)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:47:48.352334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: WARNING py.warnings [None req-25369e31-fec1-4eb6-8971-7ef40e025c82 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 08:47:48.352334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: warnings.warn(msg) Aug 26 08:47:48.352334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: Aug 26 08:47:48.354731 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25369e31-fec1-4eb6-8971-7ef40e025c82 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:47:48.467059 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-25369e31-fec1-4eb6-8971-7ef40e025c82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '313a1631eda04cc59bc71a84839ddbcd', 'request_id': 'req-25369e31-fec1-4eb6-8971-7ef40e025c82'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:47:48.474064 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-25369e31-fec1-4eb6-8971-7ef40e025c82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '313a1631eda04cc59bc71a84839ddbcd', 'request_id': 'req-25369e31-fec1-4eb6-8971-7ef40e025c82'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:47:48.486513 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-25369e31-fec1-4eb6-8971-7ef40e025c82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '313a1631eda04cc59bc71a84839ddbcd', 'request_id': 'req-25369e31-fec1-4eb6-8971-7ef40e025c82'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:47:48.536052 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-25369e31-fec1-4eb6-8971-7ef40e025c82 admin admin] The token cache is being invalidate because project 313a1631eda04cc59bc71a84839ddbcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:47:48.536561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-25369e31-fec1-4eb6-8971-7ef40e025c82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:47:48.537399 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-25369e31-fec1-4eb6-8971-7ef40e025c82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:47:48.539424 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 764/1510] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:47:48 2022] DELETE /identity/v3/projects/313a1631eda04cc59bc71a84839ddbcd => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:47:57.682518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=62 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:47:58.531873 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:49:23.705179 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46a2de-f999-4cf8-bab2-9c2474e65175 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:23.705887 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46a2de-f999-4cf8-bab2-9c2474e65175 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:23.705887 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46a2de-f999-4cf8-bab2-9c2474e65175 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:23.707158 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 747/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 08:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:49:23.714216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a872abd-bb6f-4f89-83cb-760d124c0d3f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:23.714958 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a872abd-bb6f-4f89-83cb-760d124c0d3f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:23.714958 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a872abd-bb6f-4f89-83cb-760d124c0d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:23.813807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-6a872abd-bb6f-4f89-83cb-760d124c0d3f None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:49:23.815148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-6a872abd-bb6f-4f89-83cb-760d124c0d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:49:23.956577 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 765/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 08:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:49:24.117668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46a2de-f999-4cf8-bab2-9c2474e65175 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:49:24.119997 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8e46a2de-f999-4cf8-bab2-9c2474e65175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:49:24.253148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46a2de-f999-4cf8-bab2-9c2474e65175 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:49:24.259031 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63206fb-6db9-4639-b1b5-b175a8a45643 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:49:24.259654 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63206fb-6db9-4639-b1b5-b175a8a45643 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:24.259934 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63206fb-6db9-4639-b1b5-b175a8a45643 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:24.260210 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63206fb-6db9-4639-b1b5-b175a8a45643 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:24.269812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63206fb-6db9-4639-b1b5-b175a8a45643 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:49:24.271462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63206fb-6db9-4639-b1b5-b175a8a45643 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:49:24.281213 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 748/1513] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:49:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:49:25.132405 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65301385-c0bb-4de4-8aa0-8a9e733baba6 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:25.133061 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65301385-c0bb-4de4-8aa0-8a9e733baba6 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:25.133061 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65301385-c0bb-4de4-8aa0-8a9e733baba6 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:25.134339 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 766/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 08:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:49:25.139846 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10819db-ab8b-4346-b892-99a6547fa1cc None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:25.140699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10819db-ab8b-4346-b892-99a6547fa1cc None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:25.140699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10819db-ab8b-4346-b892-99a6547fa1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:25.161322 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c10819db-ab8b-4346-b892-99a6547fa1cc None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:49:25.162545 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c10819db-ab8b-4346-b892-99a6547fa1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:49:25.171161 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 749/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 08:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:49:25.331873 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65301385-c0bb-4de4-8aa0-8a9e733baba6 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:49:25.347134 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65301385-c0bb-4de4-8aa0-8a9e733baba6 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:49:25.352772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded15a3-e329-4fb7-8f3a-882a886bd772 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:49:25.353403 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded15a3-e329-4fb7-8f3a-882a886bd772 service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:25.353679 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded15a3-e329-4fb7-8f3a-882a886bd772 service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:25.353952 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded15a3-e329-4fb7-8f3a-882a886bd772 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:25.362925 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded15a3-e329-4fb7-8f3a-882a886bd772 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:49:25.364568 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded15a3-e329-4fb7-8f3a-882a886bd772 service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:49:25.374932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 767/1516] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:49:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:49:26.183480 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b7264-ca4e-4e70-9d5e-264faa398df7 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:26.184049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b7264-ca4e-4e70-9d5e-264faa398df7 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:26.184049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b7264-ca4e-4e70-9d5e-264faa398df7 None None] PATH_INFO: `/v3` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:26.185494 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 750/1517] 10.222.0.115 () {54 vars in 926 bytes} [Fri Aug 26 08:49:26 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:49:26.194025 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d491077-efb5-4e55-b007-61982d171966 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:26.194609 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d491077-efb5-4e55-b007-61982d171966 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:26.194609 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d491077-efb5-4e55-b007-61982d171966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:26.262265 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-8d491077-efb5-4e55-b007-61982d171966 None None] MFA Rules not processed for user `0cf5e2b4f5e440a7942adea3d51cec75`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:49:26.263496 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8d491077-efb5-4e55-b007-61982d171966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:49:26.284843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:49:26.350706 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 768/1518] 10.222.0.115 () {58 vars in 1016 bytes} [Fri Aug 26 08:49:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:49:30.293649 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42e36f-e44e-48b4-a933-cda912f2533c None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:30.294242 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42e36f-e44e-48b4-a933-cda912f2533c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:30.294242 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42e36f-e44e-48b4-a933-cda912f2533c None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:30.295868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 751/1519] 10.222.0.71 () {54 vars in 961 bytes} [Fri Aug 26 08:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:49:30.301627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77b04a-643a-4f74-b404-c88d91d6544f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:30.302270 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77b04a-643a-4f74-b404-c88d91d6544f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:30.302270 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77b04a-643a-4f74-b404-c88d91d6544f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:30.324719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-8c77b04a-643a-4f74-b404-c88d91d6544f None None] MFA Rules not processed for user `0cf5e2b4f5e440a7942adea3d51cec75`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:49:30.326017 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8c77b04a-643a-4f74-b404-c88d91d6544f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:49:30.335697 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 769/1520] 10.222.0.71 () {58 vars in 1058 bytes} [Fri Aug 26 08:49:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:49:30.343934 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42e36f-e44e-48b4-a933-cda912f2533c None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:49:30.362412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c42e36f-e44e-48b4-a933-cda912f2533c None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:49:30.373338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664278ef-8877-4d04-a6c8-265f5b3bbeb6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf5e2b4f5e440a7942adea3d51cec75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:49:30.373489 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664278ef-8877-4d04-a6c8-265f5b3bbeb6 service placement] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:30.373656 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664278ef-8877-4d04-a6c8-265f5b3bbeb6 service placement] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:30.373930 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664278ef-8877-4d04-a6c8-265f5b3bbeb6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:30.383472 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664278ef-8877-4d04-a6c8-265f5b3bbeb6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:49:30.385017 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664278ef-8877-4d04-a6c8-265f5b3bbeb6 service placement] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:49:30.395137 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 752/1521] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:49:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:49:36.788622 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c56831-8dcb-484f-9476-4831a4b739dc None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:36.789302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c56831-8dcb-484f-9476-4831a4b739dc None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:36.789302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c56831-8dcb-484f-9476-4831a4b739dc None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:36.790762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 770/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 08:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:49:36.797355 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c914c-6ff2-44cd-9993-d202f6e76dd6 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:36.797993 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c914c-6ff2-44cd-9993-d202f6e76dd6 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:36.797993 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c914c-6ff2-44cd-9993-d202f6e76dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:36.821403 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-366c914c-6ff2-44cd-9993-d202f6e76dd6 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:49:36.822913 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-366c914c-6ff2-44cd-9993-d202f6e76dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:49:36.838294 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 753/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 08:49:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:49:36.999315 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c56831-8dcb-484f-9476-4831a4b739dc None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:49:37.018531 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c56831-8dcb-484f-9476-4831a4b739dc None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:49:37.027630 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a75cc-0422-4558-8214-2b92043cb137 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:49:37.028285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a75cc-0422-4558-8214-2b92043cb137 service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:37.028566 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a75cc-0422-4558-8214-2b92043cb137 service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:37.028833 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a75cc-0422-4558-8214-2b92043cb137 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:37.038201 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a75cc-0422-4558-8214-2b92043cb137 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:49:37.040699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a75cc-0422-4558-8214-2b92043cb137 service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:49:37.050992 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 771/1524] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:49:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:49:37.882254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9c25f-aa27-41eb-810e-bf7f850e2003 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:37.882873 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9c25f-aa27-41eb-810e-bf7f850e2003 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:37.882873 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f9c25f-aa27-41eb-810e-bf7f850e2003 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:37.884195 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 754/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 08:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:49:37.889868 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f97a4-286a-480d-b66d-ca8ec7781237 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:37.890365 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f97a4-286a-480d-b66d-ca8ec7781237 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:37.890477 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f97a4-286a-480d-b66d-ca8ec7781237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:37.913213 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-f93f97a4-286a-480d-b66d-ca8ec7781237 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:49:37.914535 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f93f97a4-286a-480d-b66d-ca8ec7781237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:49:37.924051 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 772/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 08:49:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:49:38.075626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9c25f-aa27-41eb-810e-bf7f850e2003 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:49:38.096446 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f9c25f-aa27-41eb-810e-bf7f850e2003 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:49:38.103604 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b9340-c1ca-443e-9e06-75d547a56b11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:49:38.104216 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b9340-c1ca-443e-9e06-75d547a56b11 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:38.104491 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b9340-c1ca-443e-9e06-75d547a56b11 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:38.104760 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b9340-c1ca-443e-9e06-75d547a56b11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:38.114254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b9340-c1ca-443e-9e06-75d547a56b11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:49:38.116250 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b9340-c1ca-443e-9e06-75d547a56b11 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:49:38.134524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 755/1527] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:49:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:49:42.023253 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47f31a-241c-4ba9-a88a-9a34886f0603 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:42.023807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47f31a-241c-4ba9-a88a-9a34886f0603 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:42.023807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47f31a-241c-4ba9-a88a-9a34886f0603 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:42.025217 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 773/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 08:49:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:49:42.031844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7923a7-3b15-4c27-b3ac-bbf7bc01b4e8 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:42.032350 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7923a7-3b15-4c27-b3ac-bbf7bc01b4e8 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:42.032490 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7923a7-3b15-4c27-b3ac-bbf7bc01b4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:42.055810 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-bd7923a7-3b15-4c27-b3ac-bbf7bc01b4e8 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:49:42.056985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-bd7923a7-3b15-4c27-b3ac-bbf7bc01b4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:49:42.065576 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 756/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 08:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:49:42.354695 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47f31a-241c-4ba9-a88a-9a34886f0603 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:49:42.357359 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-bb47f31a-241c-4ba9-a88a-9a34886f0603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:49:42.456278 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47f31a-241c-4ba9-a88a-9a34886f0603 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:49:42.460566 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded68ef-c99e-44de-b230-2c86f6d56519 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:49:42.461184 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded68ef-c99e-44de-b230-2c86f6d56519 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:42.461464 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded68ef-c99e-44de-b230-2c86f6d56519 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:42.461739 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded68ef-c99e-44de-b230-2c86f6d56519 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:42.471714 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded68ef-c99e-44de-b230-2c86f6d56519 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:49:42.473166 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded68ef-c99e-44de-b230-2c86f6d56519 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:49:42.483492 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 774/1530] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:49:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:49:44.385192 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc4232-8d3f-43a7-a460-5b77515e2ebf None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:44.385675 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc4232-8d3f-43a7-a460-5b77515e2ebf None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:44.385799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc4232-8d3f-43a7-a460-5b77515e2ebf None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:44.387198 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 757/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 08:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:49:44.393060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75490295-1b66-4ac1-9be0-186179e7de74 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:44.393508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75490295-1b66-4ac1-9be0-186179e7de74 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:44.393625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75490295-1b66-4ac1-9be0-186179e7de74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:44.418156 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-75490295-1b66-4ac1-9be0-186179e7de74 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:49:44.419413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-75490295-1b66-4ac1-9be0-186179e7de74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:49:44.428919 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 775/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 08:49:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:49:44.729484 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc4232-8d3f-43a7-a460-5b77515e2ebf None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:49:44.732691 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-72dc4232-8d3f-43a7-a460-5b77515e2ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:49:44.748970 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc4232-8d3f-43a7-a460-5b77515e2ebf None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:49:44.753073 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ae74f-572d-4c67-810b-521a2e509ca9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:49:44.753678 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ae74f-572d-4c67-810b-521a2e509ca9 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:44.753951 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ae74f-572d-4c67-810b-521a2e509ca9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:44.754217 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ae74f-572d-4c67-810b-521a2e509ca9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:44.764268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443ae74f-572d-4c67-810b-521a2e509ca9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:49:44.765791 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443ae74f-572d-4c67-810b-521a2e509ca9 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:49:44.776898 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 758/1533] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:49:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:49:48.349317 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae3d1e-cdd8-4240-9b85-3aadb6f14370 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:48.350074 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae3d1e-cdd8-4240-9b85-3aadb6f14370 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:48.350074 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae3d1e-cdd8-4240-9b85-3aadb6f14370 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:48.351533 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 776/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 08:49:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:49:48.358578 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22df3aa-17f3-47f4-9aeb-76baa0305605 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:48.359271 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22df3aa-17f3-47f4-9aeb-76baa0305605 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:48.359271 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22df3aa-17f3-47f4-9aeb-76baa0305605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:48.367861 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=81 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:49:48.381651 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c22df3aa-17f3-47f4-9aeb-76baa0305605 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:49:48.383038 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c22df3aa-17f3-47f4-9aeb-76baa0305605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:49:48.391799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 759/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 08:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:49:48.977106 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae3d1e-cdd8-4240-9b85-3aadb6f14370 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:49:48.979575 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-17ae3d1e-cdd8-4240-9b85-3aadb6f14370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:49:49.067764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:49:49.085132 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae3d1e-cdd8-4240-9b85-3aadb6f14370 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:49:49.089129 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebd91d-b0de-48fd-9c77-e1170c63840e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:49:49.090559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebd91d-b0de-48fd-9c77-e1170c63840e service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:49.090559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebd91d-b0de-48fd-9c77-e1170c63840e service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:49.090559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebd91d-b0de-48fd-9c77-e1170c63840e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:49.104113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebd91d-b0de-48fd-9c77-e1170c63840e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:49:49.104113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebd91d-b0de-48fd-9c77-e1170c63840e service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:49:49.115486 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 777/1536] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:49:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:49:51.083512 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948a976-b94c-4601-97fa-7d2b8630bf04 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:51.084122 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948a976-b94c-4601-97fa-7d2b8630bf04 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:51.084122 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948a976-b94c-4601-97fa-7d2b8630bf04 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:51.085521 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 760/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 08:49:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:49:51.091561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e04b7-c04d-4fee-93d7-893d6be78c03 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:51.092015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e04b7-c04d-4fee-93d7-893d6be78c03 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:51.092124 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e04b7-c04d-4fee-93d7-893d6be78c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:51.112818 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-cc4e04b7-c04d-4fee-93d7-893d6be78c03 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:49:51.114065 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-cc4e04b7-c04d-4fee-93d7-893d6be78c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:49:51.122959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 778/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 08:49:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:49:51.674712 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948a976-b94c-4601-97fa-7d2b8630bf04 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:49:51.676806 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c948a976-b94c-4601-97fa-7d2b8630bf04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:49:51.691563 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948a976-b94c-4601-97fa-7d2b8630bf04 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:49:51.695541 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c823d-20e8-42fc-9717-b96aade8ef80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:49:51.696150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c823d-20e8-42fc-9717-b96aade8ef80 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:49:51.696426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c823d-20e8-42fc-9717-b96aade8ef80 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:49:51.696690 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c823d-20e8-42fc-9717-b96aade8ef80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:49:51.706017 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c823d-20e8-42fc-9717-b96aade8ef80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:49:51.707516 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c823d-20e8-42fc-9717-b96aade8ef80 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:49:51.717975 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 761/1539] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:49:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:50:01.107385 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=11 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:50:01.712907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=8 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:21.213083 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b664ddd-80b6-4feb-9d20-75228c245fc4 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:21.213083 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b664ddd-80b6-4feb-9d20-75228c245fc4 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:21.213083 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b664ddd-80b6-4feb-9d20-75228c245fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:21.277099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-1b664ddd-80b6-4feb-9d20-75228c245fc4 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:21.282929 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1b664ddd-80b6-4feb-9d20-75228c245fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:21.306823 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 779/1540] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:21.333731 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c000409-e115-40e1-bc79-fd21d648f20e None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:21.334670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c000409-e115-40e1-bc79-fd21d648f20e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:21.334845 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c000409-e115-40e1-bc79-fd21d648f20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:21.421045 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-7c000409-e115-40e1-bc79-fd21d648f20e None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:21.423479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7c000409-e115-40e1-bc79-fd21d648f20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:21.454828 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 762/1541] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:21.465255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b664ddd-80b6-4feb-9d20-75228c245fc4 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:21.496296 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b664ddd-80b6-4feb-9d20-75228c245fc4 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:21.513803 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfc2d0-b336-4ffb-9889-bfa0f2121e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:21.515050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfc2d0-b336-4ffb-9889-bfa0f2121e80 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:21.515545 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfc2d0-b336-4ffb-9889-bfa0f2121e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:21.516027 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfc2d0-b336-4ffb-9889-bfa0f2121e80 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:21.517398 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cfc2d0-b336-4ffb-9889-bfa0f2121e80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:51:21.517956 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cfc2d0-b336-4ffb-9889-bfa0f2121e80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:21.521160 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cfc2d0-b336-4ffb-9889-bfa0f2121e80 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:21.558712 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 780/1542] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:51:21.570830 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c000409-e115-40e1-bc79-fd21d648f20e None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:21.594399 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c000409-e115-40e1-bc79-fd21d648f20e None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:21.600057 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e607ecec-50da-4491-82f7-d0f644dcb366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:21.600836 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607ecec-50da-4491-82f7-d0f644dcb366 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:21.601503 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607ecec-50da-4491-82f7-d0f644dcb366 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:21.601809 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607ecec-50da-4491-82f7-d0f644dcb366 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:21.602520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e607ecec-50da-4491-82f7-d0f644dcb366 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:21.605248 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e607ecec-50da-4491-82f7-d0f644dcb366 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:21.696807 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 763/1543] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:51:21 2022] POST /identity/v3/projects => generated 365 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:21.704258 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfc2d0-b336-4ffb-9889-bfa0f2121e80 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:21.736501 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfc2d0-b336-4ffb-9889-bfa0f2121e80 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:21.895975 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2230c3-e16d-4acd-af8e-c4768408c983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:21.897343 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2230c3-e16d-4acd-af8e-c4768408c983 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:21.897836 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2230c3-e16d-4acd-af8e-c4768408c983 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:21.898306 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2230c3-e16d-4acd-af8e-c4768408c983 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:21.899512 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2230c3-e16d-4acd-af8e-c4768408c983 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:21.903358 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2230c3-e16d-4acd-af8e-c4768408c983 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:21.957761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 781/1544] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:51:21 2022] POST /identity/v3/users => generated 283 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:21.963734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e607ecec-50da-4491-82f7-d0f644dcb366 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:21.984347 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e607ecec-50da-4491-82f7-d0f644dcb366 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:22.001192 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cdf7b4-e1fb-4340-a8d2-9f9299000be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:22.002617 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cdf7b4-e1fb-4340-a8d2-9f9299000be6 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:22.003008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cdf7b4-e1fb-4340-a8d2-9f9299000be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:22.003346 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cdf7b4-e1fb-4340-a8d2-9f9299000be6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:22.004030 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cdf7b4-e1fb-4340-a8d2-9f9299000be6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:22.005942 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cdf7b4-e1fb-4340-a8d2-9f9299000be6 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:22.030260 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 764/1545] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 08:51:22.039577 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2230c3-e16d-4acd-af8e-c4768408c983 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:22.068733 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2230c3-e16d-4acd-af8e-c4768408c983 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:22.086223 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092653a9-274c-450e-a75e-9c9ed9eff095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:22.087337 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092653a9-274c-450e-a75e-9c9ed9eff095 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:22.087797 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092653a9-274c-450e-a75e-9c9ed9eff095 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:22.088235 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092653a9-274c-450e-a75e-9c9ed9eff095 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:22.089290 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092653a9-274c-450e-a75e-9c9ed9eff095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:22.092380 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092653a9-274c-450e-a75e-9c9ed9eff095 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:22.121086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 782/1546] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:22 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:22.127727 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cdf7b4-e1fb-4340-a8d2-9f9299000be6 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:22.147135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cdf7b4-e1fb-4340-a8d2-9f9299000be6 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:22.162655 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355ea3e-984b-4281-bc0b-3567bb911497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:22.163550 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355ea3e-984b-4281-bc0b-3567bb911497 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:22.163954 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355ea3e-984b-4281-bc0b-3567bb911497 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:22.164235 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355ea3e-984b-4281-bc0b-3567bb911497 admin admin] PATH_INFO: `/v3/projects/ed66622c6b37441f84917768edd57f34/users/10bccdf275914aa784ff53d5a3c6abcf/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:22.202164 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8355ea3e-984b-4281-bc0b-3567bb911497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed66622c6b37441f84917768edd57f34, user_id=10bccdf275914aa784ff53d5a3c6abcf, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:22.205174 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8355ea3e-984b-4281-bc0b-3567bb911497 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:22.237972 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 765/1547] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:51:22 2022] PUT /identity/v3/projects/ed66622c6b37441f84917768edd57f34/users/10bccdf275914aa784ff53d5a3c6abcf/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:22.273777 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8cc0b-9c28-4439-9e64-73bc5faf969f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:22.275347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8cc0b-9c28-4439-9e64-73bc5faf969f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:22.276061 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8cc0b-9c28-4439-9e64-73bc5faf969f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:22.348063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-2ab8cc0b-9c28-4439-9e64-73bc5faf969f None None] MFA Rules not processed for user `10bccdf275914aa784ff53d5a3c6abcf`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:22.350387 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2ab8cc0b-9c28-4439-9e64-73bc5faf969f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:22.399016 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d523b-fa1b-4a8a-9798-701e1a88b976 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:22.399954 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d523b-fa1b-4a8a-9798-701e1a88b976 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:22.399954 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d523b-fa1b-4a8a-9798-701e1a88b976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:22.453294 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-bd8d523b-fa1b-4a8a-9798-701e1a88b976 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:22.454590 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-bd8d523b-fa1b-4a8a-9798-701e1a88b976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:22.586853 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 783/1548] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:22.596645 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab8cc0b-9c28-4439-9e64-73bc5faf969f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:22.660267 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 766/1549] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:22.673240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32daa13f-7eba-4442-9dc5-e9306d9ecf75 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:22.673718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32daa13f-7eba-4442-9dc5-e9306d9ecf75 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:22.674168 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32daa13f-7eba-4442-9dc5-e9306d9ecf75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:22.727964 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-32daa13f-7eba-4442-9dc5-e9306d9ecf75 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:22.729952 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-32daa13f-7eba-4442-9dc5-e9306d9ecf75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:22.754239 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 767/1550] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:51:22.756392 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a423ba-c271-442c-bece-24278f410477 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:22.756854 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a423ba-c271-442c-bece-24278f410477 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:22.757327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a423ba-c271-442c-bece-24278f410477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:22.776584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:22.802800 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-08a423ba-c271-442c-bece-24278f410477 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:22.804753 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-08a423ba-c271-442c-bece-24278f410477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:22.831126 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 768/1551] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:22.832597 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a423ba-c271-442c-bece-24278f410477 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:22.836154 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab8cc0b-9c28-4439-9e64-73bc5faf969f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:22.850188 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70738178-c6bf-4934-8cf8-9995c3a5259d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:22.852265 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70738178-c6bf-4934-8cf8-9995c3a5259d admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:22.852856 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70738178-c6bf-4934-8cf8-9995c3a5259d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:22.853443 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70738178-c6bf-4934-8cf8-9995c3a5259d admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:22.854646 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70738178-c6bf-4934-8cf8-9995c3a5259d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:22.857694 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a423ba-c271-442c-bece-24278f410477 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:22.860883 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: WARNING py.warnings [None req-70738178-c6bf-4934-8cf8-9995c3a5259d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 08:51:22.860883 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: warnings.warn(msg) Aug 26 08:51:22.860883 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: Aug 26 08:51:22.864514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70738178-c6bf-4934-8cf8-9995c3a5259d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:22.880583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8534ce-78dc-4980-be72-7bc213c37085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:22.881742 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8534ce-78dc-4980-be72-7bc213c37085 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:22.882020 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8534ce-78dc-4980-be72-7bc213c37085 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:22.882492 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8534ce-78dc-4980-be72-7bc213c37085 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:22.883704 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8534ce-78dc-4980-be72-7bc213c37085 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:51:22.884230 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8534ce-78dc-4980-be72-7bc213c37085 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:22.886134 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: WARNING py.warnings [None req-6c8534ce-78dc-4980-be72-7bc213c37085 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 08:51:22.886134 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: warnings.warn(msg) Aug 26 08:51:22.886134 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: Aug 26 08:51:22.892384 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8534ce-78dc-4980-be72-7bc213c37085 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:22.935852 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 769/1552] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:51:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:51:22.941161 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f04464-4ff5-4b5a-9c62-760fc411882c None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:22.941161 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f04464-4ff5-4b5a-9c62-760fc411882c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:22.941161 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f04464-4ff5-4b5a-9c62-760fc411882c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:22.947056 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 784/1553] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:51:22 2022] POST /identity/v3/projects => generated 363 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:22.949905 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d43cd6-3d96-4b47-a874-2e80c54ef900 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:22.950532 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d43cd6-3d96-4b47-a874-2e80c54ef900 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:22.951218 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d43cd6-3d96-4b47-a874-2e80c54ef900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:22.971655 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-19f04464-4ff5-4b5a-9c62-760fc411882c None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:22.973686 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-19f04464-4ff5-4b5a-9c62-760fc411882c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:22.974925 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-69d43cd6-3d96-4b47-a874-2e80c54ef900 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:22.976240 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-69d43cd6-3d96-4b47-a874-2e80c54ef900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:23.069041 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 770/1554] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:23.071359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fec706-a05b-4bb1-ac96-8d95090e2313 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:23.072018 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 785/1555] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:23.072219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fec706-a05b-4bb1-ac96-8d95090e2313 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:23.072696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fec706-a05b-4bb1-ac96-8d95090e2313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:23.074259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a62ef2-807b-4eff-bbf0-bffc81fb793b None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:23.074842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a62ef2-807b-4eff-bbf0-bffc81fb793b None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:23.075548 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a62ef2-807b-4eff-bbf0-bffc81fb793b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:23.087804 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:23.101973 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-c6a62ef2-807b-4eff-bbf0-bffc81fb793b None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:23.102491 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c6a62ef2-807b-4eff-bbf0-bffc81fb793b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:23.105431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-16fec706-a05b-4bb1-ac96-8d95090e2313 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:23.107306 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-16fec706-a05b-4bb1-ac96-8d95090e2313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:23.117473 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 786/1556] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:23.118999 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 771/1557] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:23.120106 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fec706-a05b-4bb1-ac96-8d95090e2313 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:23.131475 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f0f57-fc8b-47fa-bd30-867789c8ba02 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:23.131928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f0f57-fc8b-47fa-bd30-867789c8ba02 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:23.132283 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f0f57-fc8b-47fa-bd30-867789c8ba02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:23.133010 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fec706-a05b-4bb1-ac96-8d95090e2313 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:23.145573 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd4fe9-4d8f-4bb3-b6a8-566ab47142fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:23.146257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd4fe9-4d8f-4bb3-b6a8-566ab47142fb admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:23.146534 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd4fe9-4d8f-4bb3-b6a8-566ab47142fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:23.146805 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd4fe9-4d8f-4bb3-b6a8-566ab47142fb admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:23.147601 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd4fe9-4d8f-4bb3-b6a8-566ab47142fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:23.149593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd4fe9-4d8f-4bb3-b6a8-566ab47142fb admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:23.162594 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-6b8f0f57-fc8b-47fa-bd30-867789c8ba02 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:23.164685 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-6b8f0f57-fc8b-47fa-bd30-867789c8ba02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:23.187400 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 787/1558] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:23.189169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f0f57-fc8b-47fa-bd30-867789c8ba02 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:23.207639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 772/1559] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:51:23 2022] POST /identity/v3/projects => generated 373 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:23.207989 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f0f57-fc8b-47fa-bd30-867789c8ba02 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:23.209742 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f476f-0d24-4119-99c1-f901384773bf None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:23.210234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f476f-0d24-4119-99c1-f901384773bf None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:23.210693 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f476f-0d24-4119-99c1-f901384773bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:23.241899 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-d37f476f-0d24-4119-99c1-f901384773bf None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:23.243085 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d37f476f-0d24-4119-99c1-f901384773bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:23.310291 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba7c0f-3378-4495-a32c-f922dc23bba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:23.311919 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba7c0f-3378-4495-a32c-f922dc23bba6 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:23.312283 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba7c0f-3378-4495-a32c-f922dc23bba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:23.312707 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba7c0f-3378-4495-a32c-f922dc23bba6 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:23.313393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba7c0f-3378-4495-a32c-f922dc23bba6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:23.315480 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba7c0f-3378-4495-a32c-f922dc23bba6 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:23.316016 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 773/1560] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:23.323187 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f476f-0d24-4119-99c1-f901384773bf None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:23.339418 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f476f-0d24-4119-99c1-f901384773bf None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:23.347513 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ab687-3fdc-49ca-8d4c-689d63788c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:23.348504 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ab687-3fdc-49ca-8d4c-689d63788c7d admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:23.348968 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ab687-3fdc-49ca-8d4c-689d63788c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:23.349427 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ab687-3fdc-49ca-8d4c-689d63788c7d admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:23.350522 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ab687-3fdc-49ca-8d4c-689d63788c7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:51:23.351049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ab687-3fdc-49ca-8d4c-689d63788c7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:23.356847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 788/1561] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:51:23 2022] POST /identity/v3/users => generated 288 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:23.357114 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ab687-3fdc-49ca-8d4c-689d63788c7d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:23.357225 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe0305a-6f65-4a7c-876c-84f25f3480ed None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:23.357225 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe0305a-6f65-4a7c-876c-84f25f3480ed None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:23.357225 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe0305a-6f65-4a7c-876c-84f25f3480ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:23.391189 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 774/1562] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:51:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:51:23.392881 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c354de-5c36-4522-a716-8b93e6112e86 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:23.393234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c354de-5c36-4522-a716-8b93e6112e86 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:23.393577 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c354de-5c36-4522-a716-8b93e6112e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:23.394775 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-bfe0305a-6f65-4a7c-876c-84f25f3480ed None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:23.396689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-bfe0305a-6f65-4a7c-876c-84f25f3480ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:23.411800 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 789/1563] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:23.411800 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f82ed-f1e4-451e-adcc-369f19a572ed None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:23.411800 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f82ed-f1e4-451e-adcc-369f19a572ed None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:23.411800 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f82ed-f1e4-451e-adcc-369f19a572ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:23.420563 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-32c354de-5c36-4522-a716-8b93e6112e86 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:23.424263 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-32c354de-5c36-4522-a716-8b93e6112e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:23.435746 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 775/1564] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:23.438046 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325078dd-ff62-439e-bc09-65e1d9fb6b54 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:23.438276 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-456f82ed-f1e4-451e-adcc-369f19a572ed None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:23.438460 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325078dd-ff62-439e-bc09-65e1d9fb6b54 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:23.438637 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325078dd-ff62-439e-bc09-65e1d9fb6b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:23.440142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-456f82ed-f1e4-451e-adcc-369f19a572ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:23.453676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 790/1565] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:23.459500 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f82ed-f1e4-451e-adcc-369f19a572ed None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:23.461863 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-325078dd-ff62-439e-bc09-65e1d9fb6b54 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:23.463722 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-325078dd-ff62-439e-bc09-65e1d9fb6b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:23.470661 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f82ed-f1e4-451e-adcc-369f19a572ed None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:23.475521 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 776/1566] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:23.480649 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a95828-6a16-43b5-b3ac-091e7a32eee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:23.480649 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a95828-6a16-43b5-b3ac-091e7a32eee3 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:23.480871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b5db5-7032-465b-9f4e-c18a084228e6 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:23.480871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b5db5-7032-465b-9f4e-c18a084228e6 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:23.481077 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a95828-6a16-43b5-b3ac-091e7a32eee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:23.481194 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b5db5-7032-465b-9f4e-c18a084228e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:23.481300 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a95828-6a16-43b5-b3ac-091e7a32eee3 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:23.481300 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a95828-6a16-43b5-b3ac-091e7a32eee3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:23.483433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a95828-6a16-43b5-b3ac-091e7a32eee3 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:23.503643 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-804b5db5-7032-465b-9f4e-c18a084228e6 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:23.505758 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-804b5db5-7032-465b-9f4e-c18a084228e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:23.518641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 791/1567] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:51:23 2022] POST /identity/v3/users => generated 287 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:23.519555 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a95828-6a16-43b5-b3ac-091e7a32eee3 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:23.521876 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 777/1568] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:23.525048 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b5db5-7032-465b-9f4e-c18a084228e6 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:23.533590 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a95828-6a16-43b5-b3ac-091e7a32eee3 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:23.539357 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f8eb19-71a0-423f-9d74-8300dc09fc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:23.540023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8eb19-71a0-423f-9d74-8300dc09fc60 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:23.540291 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8eb19-71a0-423f-9d74-8300dc09fc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:23.540564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8eb19-71a0-423f-9d74-8300dc09fc60 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:23.541227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f8eb19-71a0-423f-9d74-8300dc09fc60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:23.542878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f8eb19-71a0-423f-9d74-8300dc09fc60 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:23.543198 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b5db5-7032-465b-9f4e-c18a084228e6 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:23.551086 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1892089-2f31-45c9-89ae-ca8c0e04d046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:23.552128 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1892089-2f31-45c9-89ae-ca8c0e04d046 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:23.552590 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1892089-2f31-45c9-89ae-ca8c0e04d046 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:23.553062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1892089-2f31-45c9-89ae-ca8c0e04d046 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:23.554175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1892089-2f31-45c9-89ae-ca8c0e04d046 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:23.560420 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 792/1569] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:23.560420 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f8eb19-71a0-423f-9d74-8300dc09fc60 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:23.560859 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1892089-2f31-45c9-89ae-ca8c0e04d046 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:23.574763 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f8eb19-71a0-423f-9d74-8300dc09fc60 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:23.581034 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c16a1-4d27-4292-9dab-f0a2a6abf1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:23.581897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c16a1-4d27-4292-9dab-f0a2a6abf1cd admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:23.582340 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c16a1-4d27-4292-9dab-f0a2a6abf1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:23.582673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c16a1-4d27-4292-9dab-f0a2a6abf1cd admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:23.583645 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c16a1-4d27-4292-9dab-f0a2a6abf1cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:51:23.584004 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c16a1-4d27-4292-9dab-f0a2a6abf1cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:23.586493 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c16a1-4d27-4292-9dab-f0a2a6abf1cd admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:23.590995 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:23.598859 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 778/1570] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:51:23 2022] POST /identity/v3/projects => generated 399 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:23.600048 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1892089-2f31-45c9-89ae-ca8c0e04d046 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:23.610101 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 793/1571] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:51:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:51:23.611274 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c16a1-4d27-4292-9dab-f0a2a6abf1cd admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:23.626825 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c16a1-4d27-4292-9dab-f0a2a6abf1cd admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:23.694046 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1892089-2f31-45c9-89ae-ca8c0e04d046 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:23.702421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c8aa15-b12f-479d-a142-cba5ef9f1799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:23.703162 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0e829-cd02-4830-a2e6-e5f14d149f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:23.703394 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8aa15-b12f-479d-a142-cba5ef9f1799 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:23.703853 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8aa15-b12f-479d-a142-cba5ef9f1799 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:23.704201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0e829-cd02-4830-a2e6-e5f14d149f0d admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:23.704407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8aa15-b12f-479d-a142-cba5ef9f1799 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:23.704706 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0e829-cd02-4830-a2e6-e5f14d149f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:23.705065 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0e829-cd02-4830-a2e6-e5f14d149f0d admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:23.705461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c8aa15-b12f-479d-a142-cba5ef9f1799 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:51:23.705943 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c8aa15-b12f-479d-a142-cba5ef9f1799 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:23.706225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c0e829-cd02-4830-a2e6-e5f14d149f0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:51:23.706665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c0e829-cd02-4830-a2e6-e5f14d149f0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:23.708528 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c0e829-cd02-4830-a2e6-e5f14d149f0d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:23.708874 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c8aa15-b12f-479d-a142-cba5ef9f1799 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:23.734769 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 794/1572] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:51:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:51:23.735298 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 779/1573] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:51:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:51:23.736055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c8aa15-b12f-479d-a142-cba5ef9f1799 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:23.740756 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0e829-cd02-4830-a2e6-e5f14d149f0d admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:23.758485 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c8aa15-b12f-479d-a142-cba5ef9f1799 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:23.759147 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0e829-cd02-4830-a2e6-e5f14d149f0d admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:23.763507 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d971075-b833-4fb3-abbe-75e250b71b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:23.764139 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d971075-b833-4fb3-abbe-75e250b71b73 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:23.764605 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d971075-b833-4fb3-abbe-75e250b71b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:23.765139 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d971075-b833-4fb3-abbe-75e250b71b73 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:23.766083 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d971075-b833-4fb3-abbe-75e250b71b73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:51:23.766655 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d971075-b833-4fb3-abbe-75e250b71b73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:23.767481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e3c07-7de3-4a4b-b014-c207a79cb9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:23.768508 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e3c07-7de3-4a4b-b014-c207a79cb9c6 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:23.769009 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e3c07-7de3-4a4b-b014-c207a79cb9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:23.769445 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d971075-b833-4fb3-abbe-75e250b71b73 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:23.769623 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e3c07-7de3-4a4b-b014-c207a79cb9c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:23.770732 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573e3c07-7de3-4a4b-b014-c207a79cb9c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:23.773950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573e3c07-7de3-4a4b-b014-c207a79cb9c6 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:23.787937 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 795/1574] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:51:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:51:23.789521 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d971075-b833-4fb3-abbe-75e250b71b73 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:23.800881 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 780/1575] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:23 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:23.800881 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e3c07-7de3-4a4b-b014-c207a79cb9c6 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:23.805689 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:23.809310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d971075-b833-4fb3-abbe-75e250b71b73 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:23.812619 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e3c07-7de3-4a4b-b014-c207a79cb9c6 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:23.816269 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9cac0-dafe-4a71-89a2-83e091c1dbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:23.817300 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9cac0-dafe-4a71-89a2-83e091c1dbdb admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:23.817780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9cac0-dafe-4a71-89a2-83e091c1dbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:23.818195 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9cac0-dafe-4a71-89a2-83e091c1dbdb admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:23.819386 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9cac0-dafe-4a71-89a2-83e091c1dbdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:51:23.819969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9cac0-dafe-4a71-89a2-83e091c1dbdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:23.820569 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47935bbf-4ddf-4629-b72e-623785a07205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:23.821558 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47935bbf-4ddf-4629-b72e-623785a07205 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:23.822013 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47935bbf-4ddf-4629-b72e-623785a07205 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:23.822407 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47935bbf-4ddf-4629-b72e-623785a07205 admin admin] PATH_INFO: `/v3/projects/c06db8b64b8b47bbb740dec3ae70215d/users/806dee2a3b5841eebd77c3cc4f4c2c15/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:23.824609 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9cac0-dafe-4a71-89a2-83e091c1dbdb admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:23.841229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 796/1576] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:51:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:51:23.842264 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9cac0-dafe-4a71-89a2-83e091c1dbdb admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:23.853113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9cac0-dafe-4a71-89a2-83e091c1dbdb admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:23.855576 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47935bbf-4ddf-4629-b72e-623785a07205 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c06db8b64b8b47bbb740dec3ae70215d, user_id=806dee2a3b5841eebd77c3cc4f4c2c15, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:23.858397 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ebf7b-9b4a-479d-a5b7-75d121243656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:23.859056 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ebf7b-9b4a-479d-a5b7-75d121243656 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:23.859321 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47935bbf-4ddf-4629-b72e-623785a07205 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:23.859524 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ebf7b-9b4a-479d-a5b7-75d121243656 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:23.859704 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ebf7b-9b4a-479d-a5b7-75d121243656 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:23.860427 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71ebf7b-9b4a-479d-a5b7-75d121243656 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:23.862337 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71ebf7b-9b4a-479d-a5b7-75d121243656 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:23.887550 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 781/1577] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:51:23 2022] PUT /identity/v3/projects/c06db8b64b8b47bbb740dec3ae70215d/users/806dee2a3b5841eebd77c3cc4f4c2c15/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:23.889185 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47935bbf-4ddf-4629-b72e-623785a07205 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:23.899201 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 797/1578] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:51:23 2022] POST /identity/v3/users => generated 300 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:23.900717 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ebf7b-9b4a-479d-a5b7-75d121243656 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:23.918667 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ebf7b-9b4a-479d-a5b7-75d121243656 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:24.003189 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644559c-aac8-4fae-937a-6ef4e888be46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:24.003816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644559c-aac8-4fae-937a-6ef4e888be46 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:24.004096 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644559c-aac8-4fae-937a-6ef4e888be46 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:24.004364 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644559c-aac8-4fae-937a-6ef4e888be46 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:24.005008 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644559c-aac8-4fae-937a-6ef4e888be46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:24.007143 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644559c-aac8-4fae-937a-6ef4e888be46 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:24.010193 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47935bbf-4ddf-4629-b72e-623785a07205 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:24.015503 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae338e-b4c5-40fb-8816-8ec5a6a5753d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:24.016127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae338e-b4c5-40fb-8816-8ec5a6a5753d admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:24.016416 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae338e-b4c5-40fb-8816-8ec5a6a5753d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:24.016693 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae338e-b4c5-40fb-8816-8ec5a6a5753d admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:24.017365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ae338e-b4c5-40fb-8816-8ec5a6a5753d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:24.019547 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ae338e-b4c5-40fb-8816-8ec5a6a5753d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:24.049123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 798/1579] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:51:23 2022] POST /identity/v3/projects => generated 393 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:24.050230 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644559c-aac8-4fae-937a-6ef4e888be46 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:24.066230 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 782/1580] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:51:23 2022] POST /identity/v3/projects => generated 377 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:24.066230 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae338e-b4c5-40fb-8816-8ec5a6a5753d admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:24.101170 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:24.144932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644559c-aac8-4fae-937a-6ef4e888be46 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:24.175177 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae338e-b4c5-40fb-8816-8ec5a6a5753d admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:24.183895 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947d200-e169-406a-b226-db27392db01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:24.184785 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947d200-e169-406a-b226-db27392db01c admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:24.185068 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947d200-e169-406a-b226-db27392db01c admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:24.185341 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947d200-e169-406a-b226-db27392db01c admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:24.186317 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4947d200-e169-406a-b226-db27392db01c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:24.189482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4947d200-e169-406a-b226-db27392db01c admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:24.226821 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 783/1581] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:51:24 2022] POST /identity/v3/projects => generated 373 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:24.228274 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947d200-e169-406a-b226-db27392db01c admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:24.241324 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4763a-cd1d-4652-a09f-f336aa63a7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:24.242014 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4763a-cd1d-4652-a09f-f336aa63a7bc admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:24.242290 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4763a-cd1d-4652-a09f-f336aa63a7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:24.242597 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4763a-cd1d-4652-a09f-f336aa63a7bc admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:24.243236 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c4763a-cd1d-4652-a09f-f336aa63a7bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:24.245698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c4763a-cd1d-4652-a09f-f336aa63a7bc admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:24.292297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 799/1582] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:51:24 2022] POST /identity/v3/projects => generated 387 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:24.292297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4763a-cd1d-4652-a09f-f336aa63a7bc admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:24.307723 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:24.325108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947d200-e169-406a-b226-db27392db01c admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:24.395269 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4763a-cd1d-4652-a09f-f336aa63a7bc admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:24.401759 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b680f1-62bf-4e6e-8ea9-dd61dbf424ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:24.402552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b680f1-62bf-4e6e-8ea9-dd61dbf424ac admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:24.403068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b680f1-62bf-4e6e-8ea9-dd61dbf424ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:24.403487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b680f1-62bf-4e6e-8ea9-dd61dbf424ac admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:24.404221 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b680f1-62bf-4e6e-8ea9-dd61dbf424ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:24.407120 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b680f1-62bf-4e6e-8ea9-dd61dbf424ac admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:24.418241 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf75f4a2-3ad2-457f-8259-9b4937cc27c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:24.419280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75f4a2-3ad2-457f-8259-9b4937cc27c7 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:24.419752 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75f4a2-3ad2-457f-8259-9b4937cc27c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:24.420205 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75f4a2-3ad2-457f-8259-9b4937cc27c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:24.421474 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf75f4a2-3ad2-457f-8259-9b4937cc27c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:24.423999 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf75f4a2-3ad2-457f-8259-9b4937cc27c7 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:24.438875 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 784/1583] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:24.439788 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf75f4a2-3ad2-457f-8259-9b4937cc27c7 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:24.450763 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf75f4a2-3ad2-457f-8259-9b4937cc27c7 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:24.454208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 800/1584] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:51:24 2022] POST /identity/v3/projects => generated 367 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:24.455387 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b680f1-62bf-4e6e-8ea9-dd61dbf424ac admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:24.461131 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165fb5dc-9a52-4c9c-b58a-4612e78bf0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:24.462079 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fb5dc-9a52-4c9c-b58a-4612e78bf0c9 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:24.462517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fb5dc-9a52-4c9c-b58a-4612e78bf0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:24.463176 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fb5dc-9a52-4c9c-b58a-4612e78bf0c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:24.464211 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165fb5dc-9a52-4c9c-b58a-4612e78bf0c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:24.466206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165fb5dc-9a52-4c9c-b58a-4612e78bf0c9 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:24.490290 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 785/1585] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:24.490290 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165fb5dc-9a52-4c9c-b58a-4612e78bf0c9 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:24.503364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165fb5dc-9a52-4c9c-b58a-4612e78bf0c9 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:24.537341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:24.554192 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b680f1-62bf-4e6e-8ea9-dd61dbf424ac admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:24.560252 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c4c0c-fdc3-459b-9c7c-a31b68d8483d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:24.560807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c4c0c-fdc3-459b-9c7c-a31b68d8483d admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:24.561072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c4c0c-fdc3-459b-9c7c-a31b68d8483d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:24.561516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23c4c0c-fdc3-459b-9c7c-a31b68d8483d admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:24.562184 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23c4c0c-fdc3-459b-9c7c-a31b68d8483d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:24.563907 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23c4c0c-fdc3-459b-9c7c-a31b68d8483d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:24.575763 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 801/1586] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:24.580310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c4c0c-fdc3-459b-9c7c-a31b68d8483d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:24.580667 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511e142-c6b5-4f45-82a4-502bdfab0a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:24.580667 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511e142-c6b5-4f45-82a4-502bdfab0a0b admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:24.584888 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511e142-c6b5-4f45-82a4-502bdfab0a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:24.584888 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511e142-c6b5-4f45-82a4-502bdfab0a0b admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:24.584888 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511e142-c6b5-4f45-82a4-502bdfab0a0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:24.584888 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: WARNING py.warnings [None req-9511e142-c6b5-4f45-82a4-502bdfab0a0b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 08:51:24.584888 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: warnings.warn(msg) Aug 26 08:51:24.584888 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: Aug 26 08:51:24.584888 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511e142-c6b5-4f45-82a4-502bdfab0a0b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:24.590686 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23c4c0c-fdc3-459b-9c7c-a31b68d8483d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:24.599249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2656cc2-a57c-460a-ade9-bd6ff356182b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:24.599911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2656cc2-a57c-460a-ade9-bd6ff356182b admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:24.600379 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2656cc2-a57c-460a-ade9-bd6ff356182b admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:24.600804 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2656cc2-a57c-460a-ade9-bd6ff356182b admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:24.601851 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2656cc2-a57c-460a-ade9-bd6ff356182b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:24.605629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2656cc2-a57c-460a-ade9-bd6ff356182b admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:24.608891 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 786/1587] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:51:24 2022] POST /identity/v3/users => generated 297 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:24.609718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511e142-c6b5-4f45-82a4-502bdfab0a0b admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:24.620678 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511e142-c6b5-4f45-82a4-502bdfab0a0b admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:24.628621 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d7b7a-90c7-4feb-ad55-1ed1e1f29e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:24.629725 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d7b7a-90c7-4feb-ad55-1ed1e1f29e0f admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:24.630458 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d7b7a-90c7-4feb-ad55-1ed1e1f29e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:24.630946 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d7b7a-90c7-4feb-ad55-1ed1e1f29e0f admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:24.632246 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d7b7a-90c7-4feb-ad55-1ed1e1f29e0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:24.636116 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d7b7a-90c7-4feb-ad55-1ed1e1f29e0f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:24.645256 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 802/1588] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:51:24 2022] POST /identity/v3/users => generated 289 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:24.645256 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2656cc2-a57c-460a-ade9-bd6ff356182b admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:24.654782 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2656cc2-a57c-460a-ade9-bd6ff356182b admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:24.664974 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbadc77-a2bd-4f8a-8b0d-8074f0173539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:24.665825 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbadc77-a2bd-4f8a-8b0d-8074f0173539 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:24.666418 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbadc77-a2bd-4f8a-8b0d-8074f0173539 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:24.666942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbadc77-a2bd-4f8a-8b0d-8074f0173539 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:24.667852 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbadc77-a2bd-4f8a-8b0d-8074f0173539 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:24.672005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbadc77-a2bd-4f8a-8b0d-8074f0173539 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:24.678820 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 787/1589] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:51:24 2022] POST /identity/v3/users => generated 287 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:24.678820 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d7b7a-90c7-4feb-ad55-1ed1e1f29e0f admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:24.686952 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d7b7a-90c7-4feb-ad55-1ed1e1f29e0f admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:24.695921 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5880bdc9-7df6-4e1b-82d1-1ac892d743c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:24.697225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880bdc9-7df6-4e1b-82d1-1ac892d743c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:24.697641 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880bdc9-7df6-4e1b-82d1-1ac892d743c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:24.698122 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5880bdc9-7df6-4e1b-82d1-1ac892d743c8 admin admin] PATH_INFO: `/v3/projects/88aa61636dd846f0a95be974b6b99d05/users/b6f722499abc409a8a345bcb70114eaf/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:24.714129 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 803/1590] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:51:24 2022] POST /identity/v3/users => generated 294 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:24.714129 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbadc77-a2bd-4f8a-8b0d-8074f0173539 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:24.727346 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5880bdc9-7df6-4e1b-82d1-1ac892d743c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88aa61636dd846f0a95be974b6b99d05, user_id=b6f722499abc409a8a345bcb70114eaf, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:24.728368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbadc77-a2bd-4f8a-8b0d-8074f0173539 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:24.730065 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5880bdc9-7df6-4e1b-82d1-1ac892d743c8 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:24.733522 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a4b6b5-2fbb-42c6-b4f8-f82388b13050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:24.734297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4b6b5-2fbb-42c6-b4f8-f82388b13050 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:24.734595 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4b6b5-2fbb-42c6-b4f8-f82388b13050 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:24.734926 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4b6b5-2fbb-42c6-b4f8-f82388b13050 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:24.735573 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a4b6b5-2fbb-42c6-b4f8-f82388b13050 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:24.737734 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a4b6b5-2fbb-42c6-b4f8-f82388b13050 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:24.748495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 788/1591] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:51:24 2022] PUT /identity/v3/projects/88aa61636dd846f0a95be974b6b99d05/users/b6f722499abc409a8a345bcb70114eaf/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:24.754660 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5880bdc9-7df6-4e1b-82d1-1ac892d743c8 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:24.773292 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 804/1592] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:51:24 2022] POST /identity/v3/users => generated 284 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:24.775484 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a4b6b5-2fbb-42c6-b4f8-f82388b13050 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:24.793778 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a4b6b5-2fbb-42c6-b4f8-f82388b13050 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:24.804585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:24.852943 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5880bdc9-7df6-4e1b-82d1-1ac892d743c8 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:24.861688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa461bdb-1d78-4821-9838-bfbb0ee6fd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:24.862845 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa461bdb-1d78-4821-9838-bfbb0ee6fd5a admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:24.863525 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa461bdb-1d78-4821-9838-bfbb0ee6fd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:24.863879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa461bdb-1d78-4821-9838-bfbb0ee6fd5a admin admin] PATH_INFO: `/v3/projects/c06db8b64b8b47bbb740dec3ae70215d/users/806dee2a3b5841eebd77c3cc4f4c2c15/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:24.867792 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa461bdb-1d78-4821-9838-bfbb0ee6fd5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c06db8b64b8b47bbb740dec3ae70215d, user_id=806dee2a3b5841eebd77c3cc4f4c2c15, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:24.872799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ecd3b-dce4-4826-bc78-abaf9d97a911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:24.872993 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa461bdb-1d78-4821-9838-bfbb0ee6fd5a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:24.873644 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ecd3b-dce4-4826-bc78-abaf9d97a911 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:24.874254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ecd3b-dce4-4826-bc78-abaf9d97a911 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:24.874450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ecd3b-dce4-4826-bc78-abaf9d97a911 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:24.875314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ecd3b-dce4-4826-bc78-abaf9d97a911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:24.877808 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ecd3b-dce4-4826-bc78-abaf9d97a911 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:24.886581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 789/1593] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:51:24 2022] PUT /identity/v3/projects/c06db8b64b8b47bbb740dec3ae70215d/users/806dee2a3b5841eebd77c3cc4f4c2c15/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:24.887798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa461bdb-1d78-4821-9838-bfbb0ee6fd5a admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:24.894344 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 805/1594] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:24.894344 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ecd3b-dce4-4826-bc78-abaf9d97a911 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:24.912354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ecd3b-dce4-4826-bc78-abaf9d97a911 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:25.005558 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1abac-4eed-411a-a32a-653f7da74d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:25.007147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1abac-4eed-411a-a32a-653f7da74d57 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:25.007642 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1abac-4eed-411a-a32a-653f7da74d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:25.008019 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1abac-4eed-411a-a32a-653f7da74d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:25.009297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1abac-4eed-411a-a32a-653f7da74d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:25.010441 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa461bdb-1d78-4821-9838-bfbb0ee6fd5a admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:25.011276 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1abac-4eed-411a-a32a-653f7da74d57 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:25.023052 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3420e3-d52b-4bf2-9c49-42325ff761eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:25.023694 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3420e3-d52b-4bf2-9c49-42325ff761eb admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:25.023974 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3420e3-d52b-4bf2-9c49-42325ff761eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:25.024250 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3420e3-d52b-4bf2-9c49-42325ff761eb admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:25.025020 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3420e3-d52b-4bf2-9c49-42325ff761eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:25.027129 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3420e3-d52b-4bf2-9c49-42325ff761eb admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:25.031600 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 806/1595] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:25.033598 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1abac-4eed-411a-a32a-653f7da74d57 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:25.045184 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 790/1596] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:25.050005 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3420e3-d52b-4bf2-9c49-42325ff761eb admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:25.052523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1abac-4eed-411a-a32a-653f7da74d57 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:25.060897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088183c1-120d-424e-a127-ee30c1932d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:25.061671 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3420e3-d52b-4bf2-9c49-42325ff761eb admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:25.062138 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088183c1-120d-424e-a127-ee30c1932d32 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:25.062639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088183c1-120d-424e-a127-ee30c1932d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:25.062987 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088183c1-120d-424e-a127-ee30c1932d32 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:25.063723 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088183c1-120d-424e-a127-ee30c1932d32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:25.068541 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6defd3ba-3250-4b57-b6c2-06aa720e95c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:25.069311 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6defd3ba-3250-4b57-b6c2-06aa720e95c8 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:25.069817 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6defd3ba-3250-4b57-b6c2-06aa720e95c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:25.070266 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6defd3ba-3250-4b57-b6c2-06aa720e95c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:25.071016 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088183c1-120d-424e-a127-ee30c1932d32 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:25.071561 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6defd3ba-3250-4b57-b6c2-06aa720e95c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:25.073714 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6defd3ba-3250-4b57-b6c2-06aa720e95c8 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:25.086106 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 791/1597] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:25.089018 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9f31a-31b7-4355-b4ff-a6eaa26a609a None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:25.089507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9f31a-31b7-4355-b4ff-a6eaa26a609a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:25.090070 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d9f31a-31b7-4355-b4ff-a6eaa26a609a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:25.093900 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 807/1598] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:25.093900 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088183c1-120d-424e-a127-ee30c1932d32 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:25.117857 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088183c1-120d-424e-a127-ee30c1932d32 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:25.123417 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-05d9f31a-31b7-4355-b4ff-a6eaa26a609a None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:25.124760 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-05d9f31a-31b7-4355-b4ff-a6eaa26a609a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:25.126916 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d37120f-3eaa-496e-9b8c-88b49a2b7a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:25.127572 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37120f-3eaa-496e-9b8c-88b49a2b7a1e admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:25.127857 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37120f-3eaa-496e-9b8c-88b49a2b7a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:25.128179 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37120f-3eaa-496e-9b8c-88b49a2b7a1e admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:25.129203 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d37120f-3eaa-496e-9b8c-88b49a2b7a1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:25.132702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d37120f-3eaa-496e-9b8c-88b49a2b7a1e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:25.139703 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 792/1599] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:25.141352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9f31a-31b7-4355-b4ff-a6eaa26a609a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:25.155473 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 808/1600] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:25.156166 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d9f31a-31b7-4355-b4ff-a6eaa26a609a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:25.157139 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845aa7b4-b070-477e-a9a7-32a57a858ae9 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:25.157611 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845aa7b4-b070-477e-a9a7-32a57a858ae9 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:25.158089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845aa7b4-b070-477e-a9a7-32a57a858ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:25.166472 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:25.171946 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccad8a-414d-497c-b3bd-bff40cd0d3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:25.173146 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccad8a-414d-497c-b3bd-bff40cd0d3e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:25.173569 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccad8a-414d-497c-b3bd-bff40cd0d3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:25.178054 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccad8a-414d-497c-b3bd-bff40cd0d3e0 admin admin] PATH_INFO: `/v3/projects/f9c5816dc38b4a72a372862163cc2823/users/ee946243d3984841b312e629524d8d5a/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:25.200717 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ccad8a-414d-497c-b3bd-bff40cd0d3e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9c5816dc38b4a72a372862163cc2823, user_id=ee946243d3984841b312e629524d8d5a, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:25.203774 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ccad8a-414d-497c-b3bd-bff40cd0d3e0 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:25.204761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-845aa7b4-b070-477e-a9a7-32a57a858ae9 None None] MFA Rules not processed for user `806dee2a3b5841eebd77c3cc4f4c2c15`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:25.206335 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-845aa7b4-b070-477e-a9a7-32a57a858ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:25.219307 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 793/1601] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:51:25 2022] PUT /identity/v3/projects/f9c5816dc38b4a72a372862163cc2823/users/ee946243d3984841b312e629524d8d5a/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:25.220197 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccad8a-414d-497c-b3bd-bff40cd0d3e0 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:25.317519 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccad8a-414d-497c-b3bd-bff40cd0d3e0 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:25.324336 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb8a01-176e-410a-9553-a28f6e08a13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:25.325213 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb8a01-176e-410a-9553-a28f6e08a13b admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:25.325609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb8a01-176e-410a-9553-a28f6e08a13b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:25.325904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb8a01-176e-410a-9553-a28f6e08a13b admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:25.326640 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb8a01-176e-410a-9553-a28f6e08a13b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:25.328800 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb8a01-176e-410a-9553-a28f6e08a13b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:25.338752 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 809/1602] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:25 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:25.338752 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845aa7b4-b070-477e-a9a7-32a57a858ae9 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:25.347309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 794/1603] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:25.347309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb8a01-176e-410a-9553-a28f6e08a13b admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:25.360056 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845aa7b4-b070-477e-a9a7-32a57a858ae9 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:25.362595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb8a01-176e-410a-9553-a28f6e08a13b admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:25.368489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f59a20-2e62-412f-beff-8488f80f1c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:25.369790 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f59a20-2e62-412f-beff-8488f80f1c6a admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:25.370261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f59a20-2e62-412f-beff-8488f80f1c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:25.370713 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f59a20-2e62-412f-beff-8488f80f1c6a admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:25.371792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f59a20-2e62-412f-beff-8488f80f1c6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:25.374210 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc354199-94e8-421c-9a4f-8d1f5aa5dc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:25.374543 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f59a20-2e62-412f-beff-8488f80f1c6a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:25.375514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc354199-94e8-421c-9a4f-8d1f5aa5dc3e admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:25.376020 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc354199-94e8-421c-9a4f-8d1f5aa5dc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:25.376742 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc354199-94e8-421c-9a4f-8d1f5aa5dc3e admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:25.378209 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc354199-94e8-421c-9a4f-8d1f5aa5dc3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:25.380582 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc354199-94e8-421c-9a4f-8d1f5aa5dc3e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:25.388529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 810/1604] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:25.389538 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f59a20-2e62-412f-beff-8488f80f1c6a admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:25.397105 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 795/1605] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:25.398590 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc354199-94e8-421c-9a4f-8d1f5aa5dc3e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:25.402208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f59a20-2e62-412f-beff-8488f80f1c6a admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:25.406617 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:25.408528 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cbdd79-4685-4905-90b0-1ddd2f1b952b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:25.409242 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbdd79-4685-4905-90b0-1ddd2f1b952b admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:25.409896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbdd79-4685-4905-90b0-1ddd2f1b952b admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:25.410081 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbdd79-4685-4905-90b0-1ddd2f1b952b admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:25.410815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cbdd79-4685-4905-90b0-1ddd2f1b952b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:25.413022 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cbdd79-4685-4905-90b0-1ddd2f1b952b admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:25.428856 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 811/1606] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:25.430204 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cbdd79-4685-4905-90b0-1ddd2f1b952b admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:25.443236 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cbdd79-4685-4905-90b0-1ddd2f1b952b admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:25.449267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f0d93-e507-4bc0-8712-afe62977aa17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:25.449948 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f0d93-e507-4bc0-8712-afe62977aa17 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:25.450244 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f0d93-e507-4bc0-8712-afe62977aa17 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:25.450527 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f0d93-e507-4bc0-8712-afe62977aa17 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:25.451278 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f0d93-e507-4bc0-8712-afe62977aa17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:25.453154 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f0d93-e507-4bc0-8712-afe62977aa17 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:25.466395 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 812/1607] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:25.468307 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8abb1-5306-43c2-aed9-360ad1b79d0d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:25.468495 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8abb1-5306-43c2-aed9-360ad1b79d0d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:25.468894 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8abb1-5306-43c2-aed9-360ad1b79d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:25.487422 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc354199-94e8-421c-9a4f-8d1f5aa5dc3e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:25.494626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdcdd87-bb7b-4d48-8f82-08c076cf3e7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:25.495717 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcdd87-bb7b-4d48-8f82-08c076cf3e7e service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:25.496226 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcdd87-bb7b-4d48-8f82-08c076cf3e7e service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:25.496701 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcdd87-bb7b-4d48-8f82-08c076cf3e7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:25.512956 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdcdd87-bb7b-4d48-8f82-08c076cf3e7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:25.514502 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdcdd87-bb7b-4d48-8f82-08c076cf3e7e service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:25.515995 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-9ea8abb1-5306-43c2-aed9-360ad1b79d0d None None] MFA Rules not processed for user `ee946243d3984841b312e629524d8d5a`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:25.517364 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-9ea8abb1-5306-43c2-aed9-360ad1b79d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:25.525969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 796/1608] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:51:25.526832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdcdd87-bb7b-4d48-8f82-08c076cf3e7e service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:25.538315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdcdd87-bb7b-4d48-8f82-08c076cf3e7e service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:25.543747 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec2cd8-cd00-4887-877d-02a31388c028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:25.544447 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec2cd8-cd00-4887-877d-02a31388c028 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:25.544738 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec2cd8-cd00-4887-877d-02a31388c028 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:25.545060 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec2cd8-cd00-4887-877d-02a31388c028 admin admin] PATH_INFO: `/v3/projects/76bb70b9748b4ed69bfc098327cc1234/users/d065c8a972df4b8e92996ce27e96b486/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:25.565608 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec2cd8-cd00-4887-877d-02a31388c028 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76bb70b9748b4ed69bfc098327cc1234, user_id=d065c8a972df4b8e92996ce27e96b486, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:25.570126 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec2cd8-cd00-4887-877d-02a31388c028 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:25.600506 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 797/1609] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:51:25 2022] PUT /identity/v3/projects/76bb70b9748b4ed69bfc098327cc1234/users/d065c8a972df4b8e92996ce27e96b486/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:25.600506 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec2cd8-cd00-4887-877d-02a31388c028 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:25.626950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 813/1610] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:25 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:25.628066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8abb1-5306-43c2-aed9-360ad1b79d0d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:25.651196 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8abb1-5306-43c2-aed9-360ad1b79d0d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:25.691492 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec2cd8-cd00-4887-877d-02a31388c028 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:25.697252 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cdc6da-8534-4be6-b339-c0c5debc8528 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:25.698293 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cdc6da-8534-4be6-b339-c0c5debc8528 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:25.698633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cdc6da-8534-4be6-b339-c0c5debc8528 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:25.699018 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cdc6da-8534-4be6-b339-c0c5debc8528 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:25.721153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:25.750221 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e178c-d3ba-4693-afe8-4f6c2044e0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:25.751224 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e178c-d3ba-4693-afe8-4f6c2044e0e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:25.751761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e178c-d3ba-4693-afe8-4f6c2044e0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:25.752197 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e178c-d3ba-4693-afe8-4f6c2044e0e5 admin admin] PATH_INFO: `/v3/projects/01d9fa7bc7124476aa295e69b120becd/users/3aced0ed17b946f9961d87a1d4446c37/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:25.787379 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734e178c-d3ba-4693-afe8-4f6c2044e0e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01d9fa7bc7124476aa295e69b120becd, user_id=3aced0ed17b946f9961d87a1d4446c37, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:25.792602 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: WARNING py.warnings [None req-734e178c-d3ba-4693-afe8-4f6c2044e0e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 08:51:25.792602 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: warnings.warn(msg) Aug 26 08:51:25.792602 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: Aug 26 08:51:25.796562 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734e178c-d3ba-4693-afe8-4f6c2044e0e5 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:25.810665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cdc6da-8534-4be6-b339-c0c5debc8528 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:25.813090 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cdc6da-8534-4be6-b339-c0c5debc8528 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:25.816376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 814/1611] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:51:25 2022] PUT /identity/v3/projects/01d9fa7bc7124476aa295e69b120becd/users/3aced0ed17b946f9961d87a1d4446c37/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:25.818606 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e178c-d3ba-4693-afe8-4f6c2044e0e5 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:25.843299 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 798/1612] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:25 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:51:25.845700 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cdc6da-8534-4be6-b339-c0c5debc8528 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:25.868123 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cdc6da-8534-4be6-b339-c0c5debc8528 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:25.930384 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:25.947266 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e178c-d3ba-4693-afe8-4f6c2044e0e5 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:25.960678 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf4628-f825-4264-aefe-1ff1abb49ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:25.961596 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf4628-f825-4264-aefe-1ff1abb49ec6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:25.961988 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf4628-f825-4264-aefe-1ff1abb49ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:25.962687 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf4628-f825-4264-aefe-1ff1abb49ec6 admin admin] PATH_INFO: `/v3/projects/8ed7993eba324f94a1b0f7fc23401f0f/users/63e1c5a24be04ca4abd5d5109c5763f8/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:25.987447 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf4628-f825-4264-aefe-1ff1abb49ec6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ed7993eba324f94a1b0f7fc23401f0f, user_id=63e1c5a24be04ca4abd5d5109c5763f8, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:25.991387 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf4628-f825-4264-aefe-1ff1abb49ec6 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:26.021023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 815/1613] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:51:25 2022] PUT /identity/v3/projects/8ed7993eba324f94a1b0f7fc23401f0f/users/63e1c5a24be04ca4abd5d5109c5763f8/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:26.022421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf4628-f825-4264-aefe-1ff1abb49ec6 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:26.038372 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4f860-4b44-46d7-897b-e65b6d0ac5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:26.040059 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4f860-4b44-46d7-897b-e65b6d0ac5db admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:26.041886 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4f860-4b44-46d7-897b-e65b6d0ac5db admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:26.044325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4f860-4b44-46d7-897b-e65b6d0ac5db admin admin] PATH_INFO: `/v3/projects/0ce8d281891a46a9acbefc09bcd15866/users/2351fbc5fab24fb5baaa9bac9d563a7a/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:26.086770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e4f860-4b44-46d7-897b-e65b6d0ac5db admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ce8d281891a46a9acbefc09bcd15866, user_id=2351fbc5fab24fb5baaa9bac9d563a7a, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:26.089937 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e4f860-4b44-46d7-897b-e65b6d0ac5db admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:26.130335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 799/1614] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:51:25 2022] PUT /identity/v3/projects/0ce8d281891a46a9acbefc09bcd15866/users/2351fbc5fab24fb5baaa9bac9d563a7a/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:26.131620 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b5e90-13ee-43d5-ad03-a36ed6f80e83 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:26.131932 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b5e90-13ee-43d5-ad03-a36ed6f80e83 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:26.132195 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b5e90-13ee-43d5-ad03-a36ed6f80e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:26.175211 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-132b5e90-13ee-43d5-ad03-a36ed6f80e83 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:26.176904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-132b5e90-13ee-43d5-ad03-a36ed6f80e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:26.201737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf4628-f825-4264-aefe-1ff1abb49ec6 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:26.330722 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 800/1615] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:26.331957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b5e90-13ee-43d5-ad03-a36ed6f80e83 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:26.357250 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b5e90-13ee-43d5-ad03-a36ed6f80e83 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:26.363976 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:26.369897 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471db0e8-5ed3-4194-9ed4-f1fe50988a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:26.371735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471db0e8-5ed3-4194-9ed4-f1fe50988a6d admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:26.372457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471db0e8-5ed3-4194-9ed4-f1fe50988a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:26.373360 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471db0e8-5ed3-4194-9ed4-f1fe50988a6d admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:26.374374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471db0e8-5ed3-4194-9ed4-f1fe50988a6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:26.376833 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471db0e8-5ed3-4194-9ed4-f1fe50988a6d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:26.378039 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0a9ba-a1c1-4013-bea2-2fbd7046fa55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:26.379183 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0a9ba-a1c1-4013-bea2-2fbd7046fa55 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:26.379784 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0a9ba-a1c1-4013-bea2-2fbd7046fa55 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:26.380332 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0a9ba-a1c1-4013-bea2-2fbd7046fa55 admin admin] PATH_INFO: `/v3/projects/80960e64cbd74e20a2932f80aa823ff5/users/44d60042745b43dfad23989c8ef5d689/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:26.425603 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0a9ba-a1c1-4013-bea2-2fbd7046fa55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80960e64cbd74e20a2932f80aa823ff5, user_id=44d60042745b43dfad23989c8ef5d689, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:26.427459 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 801/1616] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:51:26 2022] POST /identity/v3/projects => generated 401 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:26.429275 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556d263-7d11-42d6-9366-1a7ff8127715 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:26.429657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556d263-7d11-42d6-9366-1a7ff8127715 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:26.429986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556d263-7d11-42d6-9366-1a7ff8127715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:26.430743 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0a9ba-a1c1-4013-bea2-2fbd7046fa55 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:26.459436 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-8556d263-7d11-42d6-9366-1a7ff8127715 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:26.460710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8556d263-7d11-42d6-9366-1a7ff8127715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:26.473614 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 816/1617] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:51:26 2022] PUT /identity/v3/projects/80960e64cbd74e20a2932f80aa823ff5/users/44d60042745b43dfad23989c8ef5d689/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:26.476070 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecba0d-2339-47b4-ac6b-d721238d1774 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:26.476550 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecba0d-2339-47b4-ac6b-d721238d1774 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:26.477008 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecba0d-2339-47b4-ac6b-d721238d1774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:26.530331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-b7ecba0d-2339-47b4-ac6b-d721238d1774 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:26.531748 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b7ecba0d-2339-47b4-ac6b-d721238d1774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:26.598154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 802/1618] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:26.598714 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40218d-9437-4abd-954f-b22cc4e12e9b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:26.598874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40218d-9437-4abd-954f-b22cc4e12e9b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:26.599092 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40218d-9437-4abd-954f-b22cc4e12e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:26.626371 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 817/1619] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:26.627685 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ecba0d-2339-47b4-ac6b-d721238d1774 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:26.694032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-bf40218d-9437-4abd-954f-b22cc4e12e9b None None] MFA Rules not processed for user `2351fbc5fab24fb5baaa9bac9d563a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:26.695349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-bf40218d-9437-4abd-954f-b22cc4e12e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:26.748150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:26.807234 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ecba0d-2339-47b4-ac6b-d721238d1774 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:26.814799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00cfbc-1109-487a-a960-249a00bb039b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:26.817201 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00cfbc-1109-487a-a960-249a00bb039b service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:26.817687 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00cfbc-1109-487a-a960-249a00bb039b service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:26.818178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00cfbc-1109-487a-a960-249a00bb039b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:26.833314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb00cfbc-1109-487a-a960-249a00bb039b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:26.835895 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb00cfbc-1109-487a-a960-249a00bb039b service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:26.869092 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 803/1620] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:26.870360 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40218d-9437-4abd-954f-b22cc4e12e9b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:26.874663 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 818/1621] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:51:26.882656 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec19e13-cfdd-4953-a91b-e7e6802ffe72 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:26.883179 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec19e13-cfdd-4953-a91b-e7e6802ffe72 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:26.883647 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec19e13-cfdd-4953-a91b-e7e6802ffe72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:26.897269 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40218d-9437-4abd-954f-b22cc4e12e9b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:26.909321 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1499a4e-612c-449d-9164-371ec99b49d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:26.914803 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1499a4e-612c-449d-9164-371ec99b49d8 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:26.919992 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1499a4e-612c-449d-9164-371ec99b49d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:26.919992 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1499a4e-612c-449d-9164-371ec99b49d8 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:26.919992 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1499a4e-612c-449d-9164-371ec99b49d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:26.925956 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1499a4e-612c-449d-9164-371ec99b49d8 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:26.998026 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 804/1622] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:51:26 2022] POST /identity/v3/users => generated 307 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:26.998026 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1499a4e-612c-449d-9164-371ec99b49d8 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:27.001025 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-3ec19e13-cfdd-4953-a91b-e7e6802ffe72 None None] MFA Rules not processed for user `44d60042745b43dfad23989c8ef5d689`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:27.006265 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3ec19e13-cfdd-4953-a91b-e7e6802ffe72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:27.022926 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1499a4e-612c-449d-9164-371ec99b49d8 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:27.031610 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e8f58-32ea-4376-b521-36e9a081c8c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:27.033941 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e8f58-32ea-4376-b521-36e9a081c8c4 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:27.034262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e8f58-32ea-4376-b521-36e9a081c8c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:27.035513 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e8f58-32ea-4376-b521-36e9a081c8c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:27.071965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e8f58-32ea-4376-b521-36e9a081c8c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:27.077430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e8f58-32ea-4376-b521-36e9a081c8c4 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:27.124429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 805/1623] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:51:27.126175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e8f58-32ea-4376-b521-36e9a081c8c4 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:27.144336 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e8f58-32ea-4376-b521-36e9a081c8c4 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:27.151496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcacc9-6de9-43d2-8859-18736099c745 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:27.152738 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcacc9-6de9-43d2-8859-18736099c745 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:27.153369 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcacc9-6de9-43d2-8859-18736099c745 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:27.153871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcacc9-6de9-43d2-8859-18736099c745 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:27.155390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 819/1624] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:27.157599 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32cac1c-54cc-4ab8-800f-6710097504cc None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:27.157945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32cac1c-54cc-4ab8-800f-6710097504cc None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:27.158274 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32cac1c-54cc-4ab8-800f-6710097504cc None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:27.168020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 820/1625] 10.222.0.71 () {54 vars in 962 bytes} [Fri Aug 26 08:51:27 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:51:27.170239 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32cac1c-54cc-4ab8-800f-6710097504cc None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:27.180098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bcacc9-6de9-43d2-8859-18736099c745 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:27.182820 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bcacc9-6de9-43d2-8859-18736099c745 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:27.192914 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32cac1c-54cc-4ab8-800f-6710097504cc None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:27.201633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5660d0-ac96-401e-9dfc-788fb3f05cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:27.202737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5660d0-ac96-401e-9dfc-788fb3f05cec admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:27.203264 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5660d0-ac96-401e-9dfc-788fb3f05cec admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:27.203580 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5660d0-ac96-401e-9dfc-788fb3f05cec admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:27.204236 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 806/1626] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:51:27.204419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5660d0-ac96-401e-9dfc-788fb3f05cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:27.205735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcacc9-6de9-43d2-8859-18736099c745 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:27.208245 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5660d0-ac96-401e-9dfc-788fb3f05cec admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:27.223792 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcacc9-6de9-43d2-8859-18736099c745 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:27.231088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 821/1627] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:27.231647 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66880ed-8343-484c-884e-436eb8ea27d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:27.232254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66880ed-8343-484c-884e-436eb8ea27d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:27.232416 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66880ed-8343-484c-884e-436eb8ea27d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:27.232707 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66880ed-8343-484c-884e-436eb8ea27d5 admin admin] PATH_INFO: `/v3/users/10bccdf275914aa784ff53d5a3c6abcf` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:27.232881 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5660d0-ac96-401e-9dfc-788fb3f05cec admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:27.234223 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66880ed-8343-484c-884e-436eb8ea27d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10bccdf275914aa784ff53d5a3c6abcf)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:27.236428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66880ed-8343-484c-884e-436eb8ea27d5 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:27.245687 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:27.269680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5660d0-ac96-401e-9dfc-788fb3f05cec admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:27.282588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fae51a-c560-4f72-94c4-678e6868df10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:27.284180 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fae51a-c560-4f72-94c4-678e6868df10 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:27.286396 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fae51a-c560-4f72-94c4-678e6868df10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:27.286863 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fae51a-c560-4f72-94c4-678e6868df10 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:27.287904 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fae51a-c560-4f72-94c4-678e6868df10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:27.290346 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fae51a-c560-4f72-94c4-678e6868df10 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:27.341897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 822/1628] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:51:27 2022] POST /identity/v3/projects => generated 369 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:27.341897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fcce1-27b1-418b-a670-67a9824ecb31 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:27.341897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fcce1-27b1-418b-a670-67a9824ecb31 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:27.341897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11fcce1-27b1-418b-a670-67a9824ecb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:27.351833 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a66880ed-8343-484c-884e-436eb8ea27d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10bccdf275914aa784ff53d5a3c6abcf'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:27.371472 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a66880ed-8343-484c-884e-436eb8ea27d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10bccdf275914aa784ff53d5a3c6abcf'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:27.413065 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a66880ed-8343-484c-884e-436eb8ea27d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10bccdf275914aa784ff53d5a3c6abcf'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:27.417000 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a66880ed-8343-484c-884e-436eb8ea27d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10bccdf275914aa784ff53d5a3c6abcf'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:27.420194 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a66880ed-8343-484c-884e-436eb8ea27d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10bccdf275914aa784ff53d5a3c6abcf'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:27.446628 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e11fcce1-27b1-418b-a670-67a9824ecb31 None None] MFA Rules not processed for user `f16d1768715a41648af3fe6afadd7477`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:27.450132 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e11fcce1-27b1-418b-a670-67a9824ecb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:27.480988 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 807/1629] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:51:27 2022] DELETE /identity/v3/users/10bccdf275914aa784ff53d5a3c6abcf => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:27.483174 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66880ed-8343-484c-884e-436eb8ea27d5 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:27.486356 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a66880ed-8343-484c-884e-436eb8ea27d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:27.597375 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 823/1630] 10.222.0.71 () {58 vars in 1059 bytes} [Fri Aug 26 08:51:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:27.598907 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fcce1-27b1-418b-a670-67a9824ecb31 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:27.601271 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e11fcce1-27b1-418b-a670-67a9824ecb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:27.625329 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:27.673142 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66880ed-8343-484c-884e-436eb8ea27d5 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:27.680650 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932dfdd8-367e-4808-a8d7-cfec39112402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:27.681069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932dfdd8-367e-4808-a8d7-cfec39112402 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:27.681278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932dfdd8-367e-4808-a8d7-cfec39112402 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:27.681781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932dfdd8-367e-4808-a8d7-cfec39112402 admin admin] PATH_INFO: `/v3/projects/43fa81d1a12a47e7b85b4cc551e3b916/users/edbbf54a8f7e42f2b259c4c804b2d546/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:27.709534 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932dfdd8-367e-4808-a8d7-cfec39112402 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43fa81d1a12a47e7b85b4cc551e3b916, user_id=edbbf54a8f7e42f2b259c4c804b2d546, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:27.714042 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932dfdd8-367e-4808-a8d7-cfec39112402 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:27.747098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 808/1631] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:51:27 2022] PUT /identity/v3/projects/43fa81d1a12a47e7b85b4cc551e3b916/users/edbbf54a8f7e42f2b259c4c804b2d546/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:27.750394 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811ca64-2fb6-4c8c-91ac-01ea2f9a3eec None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:27.750950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811ca64-2fb6-4c8c-91ac-01ea2f9a3eec None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:27.751581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811ca64-2fb6-4c8c-91ac-01ea2f9a3eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:27.805569 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c811ca64-2fb6-4c8c-91ac-01ea2f9a3eec None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:27.805862 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11fcce1-27b1-418b-a670-67a9824ecb31 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:27.808484 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c811ca64-2fb6-4c8c-91ac-01ea2f9a3eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:27.921085 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dab5ce-34fe-4ba1-8b38-fa2051b38a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:27.922081 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dab5ce-34fe-4ba1-8b38-fa2051b38a92 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:27.922523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dab5ce-34fe-4ba1-8b38-fa2051b38a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:27.923152 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dab5ce-34fe-4ba1-8b38-fa2051b38a92 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:27.924383 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dab5ce-34fe-4ba1-8b38-fa2051b38a92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:27.927700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dab5ce-34fe-4ba1-8b38-fa2051b38a92 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:27.948064 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:27.965349 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 824/1632] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:51:27 2022] POST /identity/v3/users => generated 291 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:27.975568 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dab5ce-34fe-4ba1-8b38-fa2051b38a92 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:27.981965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 809/1633] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:27.983658 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811ca64-2fb6-4c8c-91ac-01ea2f9a3eec None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:28.007544 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811ca64-2fb6-4c8c-91ac-01ea2f9a3eec None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:28.019663 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca271f98-57c1-4ed2-824b-f25db7fa94fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:28.022750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca271f98-57c1-4ed2-824b-f25db7fa94fc admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:28.024794 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca271f98-57c1-4ed2-824b-f25db7fa94fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:28.025914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca271f98-57c1-4ed2-824b-f25db7fa94fc admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:28.026545 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca271f98-57c1-4ed2-824b-f25db7fa94fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:28.032237 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca271f98-57c1-4ed2-824b-f25db7fa94fc admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:28.062444 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 810/1634] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:28.069498 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca271f98-57c1-4ed2-824b-f25db7fa94fc admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:28.086369 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca271f98-57c1-4ed2-824b-f25db7fa94fc admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:28.093706 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4046756-4e9b-4e52-989e-ec16d7304ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:28.094915 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4046756-4e9b-4e52-989e-ec16d7304ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:28.095468 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4046756-4e9b-4e52-989e-ec16d7304ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:28.095849 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4046756-4e9b-4e52-989e-ec16d7304ef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:28.097204 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4046756-4e9b-4e52-989e-ec16d7304ef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:28.100265 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4046756-4e9b-4e52-989e-ec16d7304ef2 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:28.116409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dab5ce-34fe-4ba1-8b38-fa2051b38a92 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:28.123694 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6afd936-15fc-4996-9ca0-c39699622f36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:28.124361 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6afd936-15fc-4996-9ca0-c39699622f36 service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:28.124650 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6afd936-15fc-4996-9ca0-c39699622f36 service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:28.124937 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6afd936-15fc-4996-9ca0-c39699622f36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:28.126387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 811/1635] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:28.128561 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4046756-4e9b-4e52-989e-ec16d7304ef2 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:28.134513 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c6afd936-15fc-4996-9ca0-c39699622f36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:28.142521 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d4046756-4e9b-4e52-989e-ec16d7304ef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:28.299496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4046756-4e9b-4e52-989e-ec16d7304ef2 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:28.308733 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52633d-3188-42f8-a19b-4de4c0c98432 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:28.309594 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52633d-3188-42f8-a19b-4de4c0c98432 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:28.310471 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52633d-3188-42f8-a19b-4de4c0c98432 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:28.311338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52633d-3188-42f8-a19b-4de4c0c98432 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:28.323682 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6afd936-15fc-4996-9ca0-c39699622f36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:28.328927 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6afd936-15fc-4996-9ca0-c39699622f36 service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:28.337783 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e52633d-3188-42f8-a19b-4de4c0c98432 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:28.341703 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e52633d-3188-42f8-a19b-4de4c0c98432 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:28.469054 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 812/1636] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:51:28.469054 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52633d-3188-42f8-a19b-4de4c0c98432 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:28.469767 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 825/1637] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:51:28.469767 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6afd936-15fc-4996-9ca0-c39699622f36 service nova] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:28.470692 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52633d-3188-42f8-a19b-4de4c0c98432 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:28.470692 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f82e82-173f-4d33-8d24-a2fdac69e263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:28.470692 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f82e82-173f-4d33-8d24-a2fdac69e263 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:28.470692 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f82e82-173f-4d33-8d24-a2fdac69e263 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:28.477589 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f82e82-173f-4d33-8d24-a2fdac69e263 admin admin] PATH_INFO: `/v3/projects/43fa81d1a12a47e7b85b4cc551e3b916/users/edbbf54a8f7e42f2b259c4c804b2d546/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:28.477804 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6afd936-15fc-4996-9ca0-c39699622f36 service nova] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:28.477922 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f82e82-173f-4d33-8d24-a2fdac69e263 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43fa81d1a12a47e7b85b4cc551e3b916, user_id=edbbf54a8f7e42f2b259c4c804b2d546, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:28.479594 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f82e82-173f-4d33-8d24-a2fdac69e263 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:28.480095 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43370598-c02a-4e96-933f-dd9a3f3f603e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:28.481399 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43370598-c02a-4e96-933f-dd9a3f3f603e admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:28.481968 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43370598-c02a-4e96-933f-dd9a3f3f603e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:28.482461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43370598-c02a-4e96-933f-dd9a3f3f603e admin admin] PATH_INFO: `/v3/projects/533dafe3e1f84272a0de5027c880a5d9/users/103038700e26414eb766e9d6eb018c69/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:28.506962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 813/1638] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:51:28 2022] PUT /identity/v3/projects/43fa81d1a12a47e7b85b4cc551e3b916/users/edbbf54a8f7e42f2b259c4c804b2d546/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:28.524165 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0624c3-ae00-41cd-9c4b-1da93949421b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:28.524705 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0624c3-ae00-41cd-9c4b-1da93949421b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:28.525039 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0624c3-ae00-41cd-9c4b-1da93949421b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:28.533297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43370598-c02a-4e96-933f-dd9a3f3f603e admin admin] RBAC: Authorizing `identity:create_grant(project_id=533dafe3e1f84272a0de5027c880a5d9, user_id=103038700e26414eb766e9d6eb018c69, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:28.548015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43370598-c02a-4e96-933f-dd9a3f3f603e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:28.586608 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 826/1639] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:51:28 2022] PUT /identity/v3/projects/533dafe3e1f84272a0de5027c880a5d9/users/103038700e26414eb766e9d6eb018c69/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:28.588287 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43370598-c02a-4e96-933f-dd9a3f3f603e admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:28.640718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-1c0624c3-ae00-41cd-9c4b-1da93949421b None None] MFA Rules not processed for user `edbbf54a8f7e42f2b259c4c804b2d546`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:28.644112 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1c0624c3-ae00-41cd-9c4b-1da93949421b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:28.726635 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43370598-c02a-4e96-933f-dd9a3f3f603e admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:28.733136 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d407cd9-16a3-49d7-b269-8d6244c22e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:28.734702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d407cd9-16a3-49d7-b269-8d6244c22e26 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:28.735246 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d407cd9-16a3-49d7-b269-8d6244c22e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:28.735951 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d407cd9-16a3-49d7-b269-8d6244c22e26 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:28.737114 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d407cd9-16a3-49d7-b269-8d6244c22e26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:28.739410 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d407cd9-16a3-49d7-b269-8d6244c22e26 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:28.759559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 827/1640] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:28.760958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:28.761985 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d407cd9-16a3-49d7-b269-8d6244c22e26 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:28.780583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:28.790300 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d407cd9-16a3-49d7-b269-8d6244c22e26 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:28.804995 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844c9e3-3eb2-4f8f-b0a7-a3ef6057d9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:28.807229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844c9e3-3eb2-4f8f-b0a7-a3ef6057d9e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:28.808156 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844c9e3-3eb2-4f8f-b0a7-a3ef6057d9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:28.809005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844c9e3-3eb2-4f8f-b0a7-a3ef6057d9e5 admin admin] PATH_INFO: `/v3/projects/533dafe3e1f84272a0de5027c880a5d9/users/103038700e26414eb766e9d6eb018c69/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:28.814216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844c9e3-3eb2-4f8f-b0a7-a3ef6057d9e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=533dafe3e1f84272a0de5027c880a5d9, user_id=103038700e26414eb766e9d6eb018c69, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:28.821226 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844c9e3-3eb2-4f8f-b0a7-a3ef6057d9e5 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:28.860281 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 828/1641] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:51:28 2022] PUT /identity/v3/projects/533dafe3e1f84272a0de5027c880a5d9/users/103038700e26414eb766e9d6eb018c69/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:28.874519 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb754d6a-7dd8-450d-a876-7b787448e26a None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:28.875621 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb754d6a-7dd8-450d-a876-7b787448e26a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:28.876461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb754d6a-7dd8-450d-a876-7b787448e26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:28.882216 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 814/1642] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:28 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:28.904094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0624c3-ae00-41cd-9c4b-1da93949421b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:28.906720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1c0624c3-ae00-41cd-9c4b-1da93949421b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:28.962586 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-fb754d6a-7dd8-450d-a876-7b787448e26a None None] MFA Rules not processed for user `103038700e26414eb766e9d6eb018c69`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:28.966220 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-fb754d6a-7dd8-450d-a876-7b787448e26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:29.088982 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0624c3-ae00-41cd-9c4b-1da93949421b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:29.272591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 829/1643] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:28 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:29.284331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f475f-5f0a-425c-b512-451bdec16ca0 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:29.285381 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f475f-5f0a-425c-b512-451bdec16ca0 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:29.291810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f475f-5f0a-425c-b512-451bdec16ca0 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:29.304838 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 830/1644] 10.222.0.71 () {54 vars in 912 bytes} [Fri Aug 26 08:51:29 2022] GET /identity => generated 270 bytes in 36 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:51:29.307551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f475f-5f0a-425c-b512-451bdec16ca0 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:29.311816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-563f475f-5f0a-425c-b512-451bdec16ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:29.313309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb21220-8f62-48aa-9409-d543098e978f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:29.314629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb21220-8f62-48aa-9409-d543098e978f service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:29.316372 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb21220-8f62-48aa-9409-d543098e978f service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:29.317698 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb21220-8f62-48aa-9409-d543098e978f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:29.338308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f475f-5f0a-425c-b512-451bdec16ca0 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:29.348228 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638dbbbb-f579-4475-8e50-3047686d9b8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:29.349876 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dbbbb-f579-4475-8e50-3047686d9b8d service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:29.350643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dbbbb-f579-4475-8e50-3047686d9b8d service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:29.351475 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dbbbb-f579-4475-8e50-3047686d9b8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:29.391581 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638dbbbb-f579-4475-8e50-3047686d9b8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:29.394807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638dbbbb-f579-4475-8e50-3047686d9b8d service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:29.420736 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 831/1645] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:29 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:51:29.423661 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b956a2f-84a7-42c7-9c18-1c8bec04a5aa None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:29.424101 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b956a2f-84a7-42c7-9c18-1c8bec04a5aa None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:29.424653 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b956a2f-84a7-42c7-9c18-1c8bec04a5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:29.451547 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb21220-8f62-48aa-9409-d543098e978f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:29.452653 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-3b956a2f-84a7-42c7-9c18-1c8bec04a5aa None None] MFA Rules not processed for user `f16d1768715a41648af3fe6afadd7477`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:29.453909 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3b956a2f-84a7-42c7-9c18-1c8bec04a5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:29.454680 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb21220-8f62-48aa-9409-d543098e978f service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:29.482384 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 815/1646] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:28 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:51:29.485125 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb21220-8f62-48aa-9409-d543098e978f service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:29.487568 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7bb21220-8f62-48aa-9409-d543098e978f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:29.573155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 832/1647] 10.222.0.71 () {58 vars in 1009 bytes} [Fri Aug 26 08:51:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:29.595593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:29.606429 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b956a2f-84a7-42c7-9c18-1c8bec04a5aa None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:29.623876 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b956a2f-84a7-42c7-9c18-1c8bec04a5aa None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:29.629345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c681738-6dc3-4f1b-8939-9132fb529258 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:29.630155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c681738-6dc3-4f1b-8939-9132fb529258 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:29.630467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c681738-6dc3-4f1b-8939-9132fb529258 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:29.630714 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c681738-6dc3-4f1b-8939-9132fb529258 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:29.648332 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c681738-6dc3-4f1b-8939-9132fb529258 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:29.650550 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c681738-6dc3-4f1b-8939-9132fb529258 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:29.661957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb21220-8f62-48aa-9409-d543098e978f service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:29.668623 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 833/1648] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:51:29.676436 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822752bd-c201-4319-baef-8b42088b2d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:29.678132 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822752bd-c201-4319-baef-8b42088b2d38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:29.679030 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822752bd-c201-4319-baef-8b42088b2d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:29.679591 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822752bd-c201-4319-baef-8b42088b2d38 admin admin] PATH_INFO: `/v3/projects/ed66622c6b37441f84917768edd57f34` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:29.682178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822752bd-c201-4319-baef-8b42088b2d38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed66622c6b37441f84917768edd57f34)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:29.684357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: WARNING py.warnings [None req-822752bd-c201-4319-baef-8b42088b2d38 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 08:51:29.684357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: warnings.warn(msg) Aug 26 08:51:29.684357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: Aug 26 08:51:29.687181 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822752bd-c201-4319-baef-8b42088b2d38 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:29.836161 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c681738-6dc3-4f1b-8939-9132fb529258 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:29.851879 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c681738-6dc3-4f1b-8939-9132fb529258 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:29.858560 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-822752bd-c201-4319-baef-8b42088b2d38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed66622c6b37441f84917768edd57f34', 'request_id': 'req-822752bd-c201-4319-baef-8b42088b2d38'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:29.876272 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-822752bd-c201-4319-baef-8b42088b2d38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed66622c6b37441f84917768edd57f34', 'request_id': 'req-822752bd-c201-4319-baef-8b42088b2d38'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:29.915207 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-822752bd-c201-4319-baef-8b42088b2d38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed66622c6b37441f84917768edd57f34', 'request_id': 'req-822752bd-c201-4319-baef-8b42088b2d38'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:29.967210 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-822752bd-c201-4319-baef-8b42088b2d38 admin admin] The token cache is being invalidate because project ed66622c6b37441f84917768edd57f34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:51:29.967888 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-822752bd-c201-4319-baef-8b42088b2d38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:29.969141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-822752bd-c201-4319-baef-8b42088b2d38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:29.972216 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 816/1649] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:51:29 2022] DELETE /identity/v3/projects/ed66622c6b37441f84917768edd57f34 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:29.978332 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822752bd-c201-4319-baef-8b42088b2d38 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:29.984300 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-822752bd-c201-4319-baef-8b42088b2d38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:29.995756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:30.016659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad2cf9-afbf-44cf-bd4c-3d1b8953e163 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:30.017875 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad2cf9-afbf-44cf-bd4c-3d1b8953e163 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:30.018531 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad2cf9-afbf-44cf-bd4c-3d1b8953e163 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:30.019160 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad2cf9-afbf-44cf-bd4c-3d1b8953e163 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:30.023143 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-22ad2cf9-afbf-44cf-bd4c-3d1b8953e163 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:30.269870 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad2cf9-afbf-44cf-bd4c-3d1b8953e163 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:30.271498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad2cf9-afbf-44cf-bd4c-3d1b8953e163 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:30.319505 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 834/1650] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:29 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:51:30.341388 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822752bd-c201-4319-baef-8b42088b2d38 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:30.368326 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf704d-d228-4b02-8135-bb3430e8ebd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:30.371889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf704d-d228-4b02-8135-bb3430e8ebd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:30.372386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf704d-d228-4b02-8135-bb3430e8ebd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:30.380186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf704d-d228-4b02-8135-bb3430e8ebd0 admin admin] PATH_INFO: `/v3/users/806dee2a3b5841eebd77c3cc4f4c2c15` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:30.385055 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf704d-d228-4b02-8135-bb3430e8ebd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=806dee2a3b5841eebd77c3cc4f4c2c15)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:30.395375 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf704d-d228-4b02-8135-bb3430e8ebd0 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:30.587495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-35cf704d-d228-4b02-8135-bb3430e8ebd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '806dee2a3b5841eebd77c3cc4f4c2c15'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:30.609028 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-35cf704d-d228-4b02-8135-bb3430e8ebd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '806dee2a3b5841eebd77c3cc4f4c2c15'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:30.637008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-35cf704d-d228-4b02-8135-bb3430e8ebd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '806dee2a3b5841eebd77c3cc4f4c2c15'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:30.641216 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-35cf704d-d228-4b02-8135-bb3430e8ebd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '806dee2a3b5841eebd77c3cc4f4c2c15'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:30.642371 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-35cf704d-d228-4b02-8135-bb3430e8ebd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '806dee2a3b5841eebd77c3cc4f4c2c15'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:30.653432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:30.732204 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 817/1651] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:51:29 2022] DELETE /identity/v3/users/806dee2a3b5841eebd77c3cc4f4c2c15 => generated 0 bytes in 755 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:32.050691 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbd870-67b0-4dde-8d58-b4f593dfae53 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:32.051328 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbd870-67b0-4dde-8d58-b4f593dfae53 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:32.051663 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbd870-67b0-4dde-8d58-b4f593dfae53 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:32.054314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 835/1652] 10.222.0.71 () {54 vars in 962 bytes} [Fri Aug 26 08:51:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:51:32.082368 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc704f79-0abf-4ae3-b50d-a0918e939ca3 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:32.082860 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc704f79-0abf-4ae3-b50d-a0918e939ca3 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:32.083343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc704f79-0abf-4ae3-b50d-a0918e939ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:32.147379 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-dc704f79-0abf-4ae3-b50d-a0918e939ca3 None None] MFA Rules not processed for user `f16d1768715a41648af3fe6afadd7477`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:32.154439 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-dc704f79-0abf-4ae3-b50d-a0918e939ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:32.348602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 818/1653] 10.222.0.71 () {58 vars in 1059 bytes} [Fri Aug 26 08:51:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:32.363615 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd20b6-24f3-453f-b958-407a5b2a81d6 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:32.363615 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd20b6-24f3-453f-b958-407a5b2a81d6 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:32.363615 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd20b6-24f3-453f-b958-407a5b2a81d6 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:32.364899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 836/1654] 10.222.0.71 () {54 vars in 912 bytes} [Fri Aug 26 08:51:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:51:32.375615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0af623-b5f1-4f92-bacd-6a04c3243afb None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:32.375909 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0af623-b5f1-4f92-bacd-6a04c3243afb None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:32.376266 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0af623-b5f1-4f92-bacd-6a04c3243afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:32.421672 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-5c0af623-b5f1-4f92-bacd-6a04c3243afb None None] MFA Rules not processed for user `f16d1768715a41648af3fe6afadd7477`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:32.423811 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5c0af623-b5f1-4f92-bacd-6a04c3243afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:32.447499 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 819/1655] 10.222.0.71 () {58 vars in 1009 bytes} [Fri Aug 26 08:51:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:32.475689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd20b6-24f3-453f-b958-407a5b2a81d6 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:32.478655 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f4cd20b6-24f3-453f-b958-407a5b2a81d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:32.611545 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd20b6-24f3-453f-b958-407a5b2a81d6 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:32.624313 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4ded4-397a-49cb-86b7-f7f0fcca271a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:32.631270 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4ded4-397a-49cb-86b7-f7f0fcca271a service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:32.633535 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4ded4-397a-49cb-86b7-f7f0fcca271a service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:32.634213 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4ded4-397a-49cb-86b7-f7f0fcca271a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:32.651584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f4ded4-397a-49cb-86b7-f7f0fcca271a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:32.654323 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f4ded4-397a-49cb-86b7-f7f0fcca271a service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:32.680650 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 837/1656] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:51:32.999811 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0af623-b5f1-4f92-bacd-6a04c3243afb None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:33.001633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5c0af623-b5f1-4f92-bacd-6a04c3243afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:33.206507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0af623-b5f1-4f92-bacd-6a04c3243afb None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:33.223044 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94dfa1-87e7-426d-bd02-094f1d1aea61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:33.224822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94dfa1-87e7-426d-bd02-094f1d1aea61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:33.225731 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94dfa1-87e7-426d-bd02-094f1d1aea61 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:33.226591 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94dfa1-87e7-426d-bd02-094f1d1aea61 admin admin] PATH_INFO: `/v3/projects/c06db8b64b8b47bbb740dec3ae70215d` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:33.228778 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd94dfa1-87e7-426d-bd02-094f1d1aea61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c06db8b64b8b47bbb740dec3ae70215d)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:33.236260 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd94dfa1-87e7-426d-bd02-094f1d1aea61 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:33.414608 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fd94dfa1-87e7-426d-bd02-094f1d1aea61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c06db8b64b8b47bbb740dec3ae70215d', 'request_id': 'req-fd94dfa1-87e7-426d-bd02-094f1d1aea61'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:33.440587 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fd94dfa1-87e7-426d-bd02-094f1d1aea61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c06db8b64b8b47bbb740dec3ae70215d', 'request_id': 'req-fd94dfa1-87e7-426d-bd02-094f1d1aea61'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:33.455631 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fd94dfa1-87e7-426d-bd02-094f1d1aea61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c06db8b64b8b47bbb740dec3ae70215d', 'request_id': 'req-fd94dfa1-87e7-426d-bd02-094f1d1aea61'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:33.490497 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:33.514172 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fd94dfa1-87e7-426d-bd02-094f1d1aea61 admin admin] The token cache is being invalidate because project c06db8b64b8b47bbb740dec3ae70215d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:51:33.514877 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fd94dfa1-87e7-426d-bd02-094f1d1aea61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:33.517194 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fd94dfa1-87e7-426d-bd02-094f1d1aea61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:33.528634 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 820/1657] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:51:32 2022] DELETE /identity/v3/projects/c06db8b64b8b47bbb740dec3ae70215d => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:33.536089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554fe244-6131-4ad2-8647-4c5f8281a926 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:33.536592 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554fe244-6131-4ad2-8647-4c5f8281a926 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:33.537056 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554fe244-6131-4ad2-8647-4c5f8281a926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:33.645075 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-554fe244-6131-4ad2-8647-4c5f8281a926 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:33.652639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-554fe244-6131-4ad2-8647-4c5f8281a926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:33.966464 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 838/1658] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:33.982133 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23de45-4007-4dcc-9650-01204cd6b076 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:33.982431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23de45-4007-4dcc-9650-01204cd6b076 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:33.982820 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23de45-4007-4dcc-9650-01204cd6b076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:34.102689 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-df23de45-4007-4dcc-9650-01204cd6b076 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:34.104222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-df23de45-4007-4dcc-9650-01204cd6b076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:34.154523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 821/1659] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:34.157987 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554fe244-6131-4ad2-8647-4c5f8281a926 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:34.223471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554fe244-6131-4ad2-8647-4c5f8281a926 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:34.274188 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d5447a-f9e4-464c-84e1-054eac9f3b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:34.276079 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5447a-f9e4-464c-84e1-054eac9f3b91 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:34.276842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5447a-f9e4-464c-84e1-054eac9f3b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:34.277615 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d5447a-f9e4-464c-84e1-054eac9f3b91 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:34.280560 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d5447a-f9e4-464c-84e1-054eac9f3b91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:51:34.281429 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d5447a-f9e4-464c-84e1-054eac9f3b91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:34.289406 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d5447a-f9e4-464c-84e1-054eac9f3b91 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:34.321969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 839/1660] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:51:34.328755 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23de45-4007-4dcc-9650-01204cd6b076 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:34.381804 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23de45-4007-4dcc-9650-01204cd6b076 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:34.410872 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb9561-b48e-443e-824a-4774db7bb11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:34.414428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb9561-b48e-443e-824a-4774db7bb11b admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:34.414956 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb9561-b48e-443e-824a-4774db7bb11b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:34.415865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb9561-b48e-443e-824a-4774db7bb11b admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:34.426757 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb9561-b48e-443e-824a-4774db7bb11b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:34.432095 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb9561-b48e-443e-824a-4774db7bb11b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:34.596568 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d5447a-f9e4-464c-84e1-054eac9f3b91 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:34.607198 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 822/1661] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:51:34 2022] POST /identity/v3/projects => generated 357 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:51:34.627295 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d5447a-f9e4-464c-84e1-054eac9f3b91 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:34.843472 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:35.051204 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ed489-9772-434f-8605-79af559c3987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:35.051990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ed489-9772-434f-8605-79af559c3987 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:35.052960 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ed489-9772-434f-8605-79af559c3987 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:35.053243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ed489-9772-434f-8605-79af559c3987 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:35.054451 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ed489-9772-434f-8605-79af559c3987 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:35.067447 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ed489-9772-434f-8605-79af559c3987 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:35.076209 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb9561-b48e-443e-824a-4774db7bb11b admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:35.079408 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-bacb9561-b48e-443e-824a-4774db7bb11b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:35.119678 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 840/1662] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:51:34 2022] POST /identity/v3/users => generated 279 bytes in 524 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:51:35.128351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ed489-9772-434f-8605-79af559c3987 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:35.154657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb9561-b48e-443e-824a-4774db7bb11b admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:35.163103 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ed489-9772-434f-8605-79af559c3987 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:35.189574 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea023b-1230-4def-9ff2-35cc57b4dbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:35.191043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea023b-1230-4def-9ff2-35cc57b4dbc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:35.191544 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea023b-1230-4def-9ff2-35cc57b4dbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:35.192035 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea023b-1230-4def-9ff2-35cc57b4dbc1 admin admin] PATH_INFO: `/v3/users/2351fbc5fab24fb5baaa9bac9d563a7a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:35.192927 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0c189-6c45-458c-b974-937c98001dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:35.194533 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0c189-6c45-458c-b974-937c98001dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:35.195694 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0c189-6c45-458c-b974-937c98001dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:35.197922 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0c189-6c45-458c-b974-937c98001dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:35.198041 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ea023b-1230-4def-9ff2-35cc57b4dbc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2351fbc5fab24fb5baaa9bac9d563a7a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:35.201287 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a0c189-6c45-458c-b974-937c98001dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:35.210012 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a0c189-6c45-458c-b974-937c98001dd4 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:35.211342 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ea023b-1230-4def-9ff2-35cc57b4dbc1 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:35.270971 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 841/1663] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:35.273137 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0c189-6c45-458c-b974-937c98001dd4 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:35.325731 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0c189-6c45-458c-b974-937c98001dd4 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:35.347640 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e19b07-93ab-4edc-9ae4-f99f822d5caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:35.349024 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e19b07-93ab-4edc-9ae4-f99f822d5caf admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:35.350006 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e19b07-93ab-4edc-9ae4-f99f822d5caf admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:35.350838 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e19b07-93ab-4edc-9ae4-f99f822d5caf admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:35.352630 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e19b07-93ab-4edc-9ae4-f99f822d5caf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:35.365497 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e19b07-93ab-4edc-9ae4-f99f822d5caf admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:35.436185 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 842/1664] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:35.444119 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e19b07-93ab-4edc-9ae4-f99f822d5caf admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:35.450779 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-81ea023b-1230-4def-9ff2-35cc57b4dbc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2351fbc5fab24fb5baaa9bac9d563a7a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:35.480210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e19b07-93ab-4edc-9ae4-f99f822d5caf admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:35.486162 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-81ea023b-1230-4def-9ff2-35cc57b4dbc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2351fbc5fab24fb5baaa9bac9d563a7a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:35.501685 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b1ddb-51d3-4c70-81b8-6e668a9afdc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:35.503062 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b1ddb-51d3-4c70-81b8-6e668a9afdc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:35.503956 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b1ddb-51d3-4c70-81b8-6e668a9afdc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:35.504552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b1ddb-51d3-4c70-81b8-6e668a9afdc8 admin admin] PATH_INFO: `/v3/projects/7bccc002c73847b3b688c5eee1cef8b0/users/7b18e70d8d114e62996b0ed0b96f55ea/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:35.535448 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-81ea023b-1230-4def-9ff2-35cc57b4dbc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2351fbc5fab24fb5baaa9bac9d563a7a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:35.536655 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-81ea023b-1230-4def-9ff2-35cc57b4dbc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2351fbc5fab24fb5baaa9bac9d563a7a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:35.543277 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-81ea023b-1230-4def-9ff2-35cc57b4dbc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2351fbc5fab24fb5baaa9bac9d563a7a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:35.559573 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b1ddb-51d3-4c70-81b8-6e668a9afdc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bccc002c73847b3b688c5eee1cef8b0, user_id=7b18e70d8d114e62996b0ed0b96f55ea, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:35.565156 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b1ddb-51d3-4c70-81b8-6e668a9afdc8 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:35.622827 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 843/1665] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:51:35 2022] PUT /identity/v3/projects/7bccc002c73847b3b688c5eee1cef8b0/users/7b18e70d8d114e62996b0ed0b96f55ea/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:51:35.650507 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ab409-79dc-409f-a35f-d3f5aaf6d8cf None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:35.653005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ab409-79dc-409f-a35f-d3f5aaf6d8cf None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:35.653601 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ab409-79dc-409f-a35f-d3f5aaf6d8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:35.680638 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 823/1666] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:51:35 2022] DELETE /identity/v3/users/2351fbc5fab24fb5baaa9bac9d563a7a => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:35.687047 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1211e-d2d4-473c-8113-2abb6cfd8a4b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:35.687826 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1211e-d2d4-473c-8113-2abb6cfd8a4b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:35.688214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1211e-d2d4-473c-8113-2abb6cfd8a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:35.765466 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-723ab409-79dc-409f-a35f-d3f5aaf6d8cf None None] MFA Rules not processed for user `7b18e70d8d114e62996b0ed0b96f55ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:35.767626 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-723ab409-79dc-409f-a35f-d3f5aaf6d8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:35.792865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-7cd1211e-d2d4-473c-8113-2abb6cfd8a4b None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:35.804678 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7cd1211e-d2d4-473c-8113-2abb6cfd8a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:36.146347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 844/1667] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:35 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 492 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:51:36.148598 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ab409-79dc-409f-a35f-d3f5aaf6d8cf None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:36.152354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-723ab409-79dc-409f-a35f-d3f5aaf6d8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:36.312793 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 824/1668] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 632 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:36.522208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ab409-79dc-409f-a35f-d3f5aaf6d8cf None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:36.542694 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59442377-428f-48be-813a-52767e1f2997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:36.543678 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59442377-428f-48be-813a-52767e1f2997 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:36.544296 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59442377-428f-48be-813a-52767e1f2997 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:36.544903 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59442377-428f-48be-813a-52767e1f2997 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:36.545895 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59442377-428f-48be-813a-52767e1f2997 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:36.548634 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59442377-428f-48be-813a-52767e1f2997 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:36.555861 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1211e-d2d4-473c-8113-2abb6cfd8a4b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:36.558336 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7cd1211e-d2d4-473c-8113-2abb6cfd8a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:36.586732 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:36.680390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 845/1669] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:51:36 2022] POST /identity/v3/projects => generated 357 bytes in 533 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:51:36.682317 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59442377-428f-48be-813a-52767e1f2997 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:36.807156 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:36.870438 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1211e-d2d4-473c-8113-2abb6cfd8a4b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:36.891761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59442377-428f-48be-813a-52767e1f2997 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:36.911206 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d3b11-4f61-4cba-b2bd-fd9ff1c787bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:36.912076 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d3b11-4f61-4cba-b2bd-fd9ff1c787bb admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:36.912475 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d3b11-4f61-4cba-b2bd-fd9ff1c787bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:36.912769 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d3b11-4f61-4cba-b2bd-fd9ff1c787bb admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:36.913526 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d3b11-4f61-4cba-b2bd-fd9ff1c787bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:36.919662 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d3b11-4f61-4cba-b2bd-fd9ff1c787bb admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:36.986169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 846/1670] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:51:36 2022] POST /identity/v3/users => generated 285 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:36.993698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d3b11-4f61-4cba-b2bd-fd9ff1c787bb admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:37.032246 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d3b11-4f61-4cba-b2bd-fd9ff1c787bb admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:37.052715 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae8591c-c362-43eb-bc5d-182b6fce4292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:37.053778 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8591c-c362-43eb-bc5d-182b6fce4292 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:37.054506 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8591c-c362-43eb-bc5d-182b6fce4292 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:37.055198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8591c-c362-43eb-bc5d-182b6fce4292 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:37.056669 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae8591c-c362-43eb-bc5d-182b6fce4292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:37.059581 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae8591c-c362-43eb-bc5d-182b6fce4292 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:37.080547 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c6975-f67c-4820-958c-6df9e8dc3ad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:37.082143 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c6975-f67c-4820-958c-6df9e8dc3ad6 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:37.082782 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c6975-f67c-4820-958c-6df9e8dc3ad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:37.084485 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c6975-f67c-4820-958c-6df9e8dc3ad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:37.096959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 847/1671] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:37.098252 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae8591c-c362-43eb-bc5d-182b6fce4292 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:37.109815 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c6975-f67c-4820-958c-6df9e8dc3ad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:37.115653 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c6975-f67c-4820-958c-6df9e8dc3ad6 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:37.116782 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-9ae8591c-c362-43eb-bc5d-182b6fce4292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:37.153626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 825/1672] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:51:37.154612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae8591c-c362-43eb-bc5d-182b6fce4292 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:37.156906 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c6975-f67c-4820-958c-6df9e8dc3ad6 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:37.173790 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c64f70-ba56-4f3b-86d9-99d177e8e003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:37.175007 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c64f70-ba56-4f3b-86d9-99d177e8e003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:37.175432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c64f70-ba56-4f3b-86d9-99d177e8e003 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:37.175845 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c64f70-ba56-4f3b-86d9-99d177e8e003 admin admin] PATH_INFO: `/v3/users/44d60042745b43dfad23989c8ef5d689` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:37.177355 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c64f70-ba56-4f3b-86d9-99d177e8e003 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44d60042745b43dfad23989c8ef5d689)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:37.178693 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: WARNING py.warnings [None req-e5c64f70-ba56-4f3b-86d9-99d177e8e003 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 08:51:37.178693 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: warnings.warn(msg) Aug 26 08:51:37.178693 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: Aug 26 08:51:37.180290 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c64f70-ba56-4f3b-86d9-99d177e8e003 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:37.183017 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c6975-f67c-4820-958c-6df9e8dc3ad6 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:37.194560 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3cc925-1ad4-49b9-9937-72347229e102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:37.197141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cc925-1ad4-49b9-9937-72347229e102 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:37.198637 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cc925-1ad4-49b9-9937-72347229e102 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:37.199499 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cc925-1ad4-49b9-9937-72347229e102 admin admin] PATH_INFO: `/v3/projects/b1b60eab32a94271977129bf7af6b1b7/users/b63b0e53a7c2490999ea1f6c0a63c74f/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:37.251546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3cc925-1ad4-49b9-9937-72347229e102 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1b60eab32a94271977129bf7af6b1b7, user_id=b63b0e53a7c2490999ea1f6c0a63c74f, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:37.265751 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3cc925-1ad4-49b9-9937-72347229e102 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:37.314554 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 826/1673] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:51:37 2022] PUT /identity/v3/projects/b1b60eab32a94271977129bf7af6b1b7/users/b63b0e53a7c2490999ea1f6c0a63c74f/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:37.316735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3cc925-1ad4-49b9-9937-72347229e102 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:37.413751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e5c64f70-ba56-4f3b-86d9-99d177e8e003 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44d60042745b43dfad23989c8ef5d689'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:37.497527 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e5c64f70-ba56-4f3b-86d9-99d177e8e003 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44d60042745b43dfad23989c8ef5d689'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:37.509507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3cc925-1ad4-49b9-9937-72347229e102 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:37.518511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdddc917-c4c7-45ff-8461-7f5ca88e09a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:37.519062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdddc917-c4c7-45ff-8461-7f5ca88e09a8 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:37.519497 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdddc917-c4c7-45ff-8461-7f5ca88e09a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:37.520648 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdddc917-c4c7-45ff-8461-7f5ca88e09a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:37.521852 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdddc917-c4c7-45ff-8461-7f5ca88e09a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:37.523869 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdddc917-c4c7-45ff-8461-7f5ca88e09a8 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:37.525314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e5c64f70-ba56-4f3b-86d9-99d177e8e003 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44d60042745b43dfad23989c8ef5d689'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:37.543675 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 827/1674] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:37.548331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdddc917-c4c7-45ff-8461-7f5ca88e09a8 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:37.564747 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdddc917-c4c7-45ff-8461-7f5ca88e09a8 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:37.569648 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e5c64f70-ba56-4f3b-86d9-99d177e8e003 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44d60042745b43dfad23989c8ef5d689'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:37.572043 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e5c64f70-ba56-4f3b-86d9-99d177e8e003 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44d60042745b43dfad23989c8ef5d689'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:37.573555 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e28e6b8-5352-406e-a30d-ca2cedf47116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:37.574431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28e6b8-5352-406e-a30d-ca2cedf47116 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:37.574819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28e6b8-5352-406e-a30d-ca2cedf47116 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:37.575735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e28e6b8-5352-406e-a30d-ca2cedf47116 admin admin] PATH_INFO: `/v3/projects/b1b60eab32a94271977129bf7af6b1b7/users/b63b0e53a7c2490999ea1f6c0a63c74f/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:37.579502 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e28e6b8-5352-406e-a30d-ca2cedf47116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1b60eab32a94271977129bf7af6b1b7, user_id=b63b0e53a7c2490999ea1f6c0a63c74f, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:37.583273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e28e6b8-5352-406e-a30d-ca2cedf47116 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:37.596103 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 848/1675] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:51:37 2022] DELETE /identity/v3/users/44d60042745b43dfad23989c8ef5d689 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:37.598055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac34b7b-888c-4efd-b230-faf56d8b7b17 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:37.598526 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac34b7b-888c-4efd-b230-faf56d8b7b17 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:37.598989 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac34b7b-888c-4efd-b230-faf56d8b7b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:37.612635 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 828/1676] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:51:37 2022] PUT /identity/v3/projects/b1b60eab32a94271977129bf7af6b1b7/users/b63b0e53a7c2490999ea1f6c0a63c74f/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:51:37.627402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-7ac34b7b-888c-4efd-b230-faf56d8b7b17 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:37.629360 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0f8ce-9ea6-403d-9a19-3b36a131fb3a None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:37.629697 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-7ac34b7b-888c-4efd-b230-faf56d8b7b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:37.631154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0f8ce-9ea6-403d-9a19-3b36a131fb3a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:37.631776 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0f8ce-9ea6-403d-9a19-3b36a131fb3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:37.707429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-4eb0f8ce-9ea6-403d-9a19-3b36a131fb3a None None] MFA Rules not processed for user `b63b0e53a7c2490999ea1f6c0a63c74f`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:37.709217 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4eb0f8ce-9ea6-403d-9a19-3b36a131fb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:37.739084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:37.835325 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 849/1677] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:37.854317 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac34b7b-888c-4efd-b230-faf56d8b7b17 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:37.854759 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:37.858101 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-7ac34b7b-888c-4efd-b230-faf56d8b7b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:37.921481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 829/1678] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:37 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:37.949173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0f8ce-9ea6-403d-9a19-3b36a131fb3a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:37.950908 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4eb0f8ce-9ea6-403d-9a19-3b36a131fb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:38.096102 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0f8ce-9ea6-403d-9a19-3b36a131fb3a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:38.113077 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1302c-fa0b-4e61-8f1c-1f0147d64013 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:38.113768 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1302c-fa0b-4e61-8f1c-1f0147d64013 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:38.114054 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1302c-fa0b-4e61-8f1c-1f0147d64013 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:38.114333 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1302c-fa0b-4e61-8f1c-1f0147d64013 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:38.143678 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf1302c-fa0b-4e61-8f1c-1f0147d64013 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:38.145118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac34b7b-888c-4efd-b230-faf56d8b7b17 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:38.145242 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf1302c-fa0b-4e61-8f1c-1f0147d64013 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:38.159122 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da404cf3-2e00-4f07-8eeb-60970d7372b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:38.159916 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da404cf3-2e00-4f07-8eeb-60970d7372b5 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:38.160274 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da404cf3-2e00-4f07-8eeb-60970d7372b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:38.160533 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da404cf3-2e00-4f07-8eeb-60970d7372b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:38.181311 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 830/1679] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:37 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:51:38.225646 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da404cf3-2e00-4f07-8eeb-60970d7372b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:38.230907 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da404cf3-2e00-4f07-8eeb-60970d7372b5 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:38.272732 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 850/1680] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:51:38.415422 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1302c-fa0b-4e61-8f1c-1f0147d64013 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:38.445927 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1302c-fa0b-4e61-8f1c-1f0147d64013 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:38.468252 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da404cf3-2e00-4f07-8eeb-60970d7372b5 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:38.471795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-da404cf3-2e00-4f07-8eeb-60970d7372b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:38.477373 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06356ef3-eb71-4e71-932d-f1c1db12005a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:38.478059 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06356ef3-eb71-4e71-932d-f1c1db12005a service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:38.478352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06356ef3-eb71-4e71-932d-f1c1db12005a service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:38.478671 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06356ef3-eb71-4e71-932d-f1c1db12005a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:38.492943 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-06356ef3-eb71-4e71-932d-f1c1db12005a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:38.499529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da404cf3-2e00-4f07-8eeb-60970d7372b5 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:38.517241 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0f86d-258e-468b-8143-acbef770ee96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:38.518032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0f86d-258e-468b-8143-acbef770ee96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:38.518425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0f86d-258e-468b-8143-acbef770ee96 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:38.518711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0f86d-258e-468b-8143-acbef770ee96 admin admin] PATH_INFO: `/v3/users/ee946243d3984841b312e629524d8d5a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:38.523849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe0f86d-258e-468b-8143-acbef770ee96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee946243d3984841b312e629524d8d5a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:38.526567 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe0f86d-258e-468b-8143-acbef770ee96 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:38.651233 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8fe0f86d-258e-468b-8143-acbef770ee96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee946243d3984841b312e629524d8d5a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:38.682107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06356ef3-eb71-4e71-932d-f1c1db12005a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:38.684300 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06356ef3-eb71-4e71-932d-f1c1db12005a service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:38.703348 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8fe0f86d-258e-468b-8143-acbef770ee96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee946243d3984841b312e629524d8d5a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:38.718101 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8fe0f86d-258e-468b-8143-acbef770ee96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee946243d3984841b312e629524d8d5a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:38.741471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8fe0f86d-258e-468b-8143-acbef770ee96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee946243d3984841b312e629524d8d5a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:38.742698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8fe0f86d-258e-468b-8143-acbef770ee96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee946243d3984841b312e629524d8d5a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:38.744932 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 831/1681] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:38 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:51:38.765104 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 851/1682] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:51:38 2022] DELETE /identity/v3/users/ee946243d3984841b312e629524d8d5a => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:38.770445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e07966-133f-46aa-82f1-254239ac8fbf None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:38.771235 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e07966-133f-46aa-82f1-254239ac8fbf None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:38.771744 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e07966-133f-46aa-82f1-254239ac8fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:38.828724 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-58e07966-133f-46aa-82f1-254239ac8fbf None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:38.839285 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-58e07966-133f-46aa-82f1-254239ac8fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:39.150337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 832/1683] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:39.286821 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0f86d-258e-468b-8143-acbef770ee96 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:39.292596 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8fe0f86d-258e-468b-8143-acbef770ee96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:39.516563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0f86d-258e-468b-8143-acbef770ee96 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:39.554739 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634b677-65b4-4c41-aa7f-258c5697c118 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:39.558070 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634b677-65b4-4c41-aa7f-258c5697c118 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:39.559695 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634b677-65b4-4c41-aa7f-258c5697c118 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:39.560150 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634b677-65b4-4c41-aa7f-258c5697c118 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:39.636268 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:39.653466 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634b677-65b4-4c41-aa7f-258c5697c118 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:39.660142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634b677-65b4-4c41-aa7f-258c5697c118 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:39.715524 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 852/1684] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:51:40.562861 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e07966-133f-46aa-82f1-254239ac8fbf None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:40.575495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-58e07966-133f-46aa-82f1-254239ac8fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:40.617880 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e07966-133f-46aa-82f1-254239ac8fbf None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:40.636986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8414dc76-8da7-4a4d-8a4b-995bb41e7ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:40.638498 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414dc76-8da7-4a4d-8a4b-995bb41e7ee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:40.638498 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414dc76-8da7-4a4d-8a4b-995bb41e7ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:40.638498 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414dc76-8da7-4a4d-8a4b-995bb41e7ee5 admin admin] PATH_INFO: `/v3/projects/0ce8d281891a46a9acbefc09bcd15866` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:40.644983 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8414dc76-8da7-4a4d-8a4b-995bb41e7ee5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ce8d281891a46a9acbefc09bcd15866)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:40.650617 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8414dc76-8da7-4a4d-8a4b-995bb41e7ee5 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:40.769665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:40.910075 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-8414dc76-8da7-4a4d-8a4b-995bb41e7ee5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ce8d281891a46a9acbefc09bcd15866', 'request_id': 'req-8414dc76-8da7-4a4d-8a4b-995bb41e7ee5'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:40.933186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-8414dc76-8da7-4a4d-8a4b-995bb41e7ee5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ce8d281891a46a9acbefc09bcd15866', 'request_id': 'req-8414dc76-8da7-4a4d-8a4b-995bb41e7ee5'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:40.974053 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-8414dc76-8da7-4a4d-8a4b-995bb41e7ee5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ce8d281891a46a9acbefc09bcd15866', 'request_id': 'req-8414dc76-8da7-4a4d-8a4b-995bb41e7ee5'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:41.020075 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-8414dc76-8da7-4a4d-8a4b-995bb41e7ee5 admin admin] The token cache is being invalidate because project 0ce8d281891a46a9acbefc09bcd15866 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:51:41.020782 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-8414dc76-8da7-4a4d-8a4b-995bb41e7ee5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:41.022622 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-8414dc76-8da7-4a4d-8a4b-995bb41e7ee5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:41.029827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 833/1685] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:51:40 2022] DELETE /identity/v3/projects/0ce8d281891a46a9acbefc09bcd15866 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:41.036761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c08214-2fbd-4d55-ba3a-aacbc133004c None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:41.038359 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c08214-2fbd-4d55-ba3a-aacbc133004c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:41.039244 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c08214-2fbd-4d55-ba3a-aacbc133004c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:41.086756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-93c08214-2fbd-4d55-ba3a-aacbc133004c None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:41.092914 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-93c08214-2fbd-4d55-ba3a-aacbc133004c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:41.236749 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cac50-c2ce-40f4-a9b4-c2eab5d26021 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:41.237092 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 853/1686] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:41.237261 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cac50-c2ce-40f4-a9b4-c2eab5d26021 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:41.237648 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cac50-c2ce-40f4-a9b4-c2eab5d26021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:41.288528 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-1b6cac50-c2ce-40f4-a9b4-c2eab5d26021 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:41.290607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1b6cac50-c2ce-40f4-a9b4-c2eab5d26021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:41.321992 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 834/1687] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:41.328097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c08214-2fbd-4d55-ba3a-aacbc133004c None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:41.344582 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c08214-2fbd-4d55-ba3a-aacbc133004c None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:41.352036 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501e5b7-59ac-4f50-b793-251bf800e20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:41.353447 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501e5b7-59ac-4f50-b793-251bf800e20a admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:41.359754 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501e5b7-59ac-4f50-b793-251bf800e20a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:41.360842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501e5b7-59ac-4f50-b793-251bf800e20a admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:41.362535 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9501e5b7-59ac-4f50-b793-251bf800e20a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:51:41.363381 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9501e5b7-59ac-4f50-b793-251bf800e20a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:41.366317 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9501e5b7-59ac-4f50-b793-251bf800e20a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:41.391740 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 854/1688] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:51:41.393230 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6cac50-c2ce-40f4-a9b4-c2eab5d26021 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:41.414650 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6cac50-c2ce-40f4-a9b4-c2eab5d26021 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:41.427012 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988205c-4656-465a-9616-8fa8b0303d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:41.428043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988205c-4656-465a-9616-8fa8b0303d51 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:41.428502 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988205c-4656-465a-9616-8fa8b0303d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:41.429393 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9988205c-4656-465a-9616-8fa8b0303d51 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:41.431761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9988205c-4656-465a-9616-8fa8b0303d51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:41.434421 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9988205c-4656-465a-9616-8fa8b0303d51 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:41.520136 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 835/1689] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:51:41 2022] POST /identity/v3/projects => generated 371 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:41.521957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501e5b7-59ac-4f50-b793-251bf800e20a admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:41.545742 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501e5b7-59ac-4f50-b793-251bf800e20a admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:41.717334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4bed2-5671-4b5e-a23e-73e1b2a01889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:41.725916 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988205c-4656-465a-9616-8fa8b0303d51 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:41.728691 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9988205c-4656-465a-9616-8fa8b0303d51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:41.734112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4bed2-5671-4b5e-a23e-73e1b2a01889 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:41.735694 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4bed2-5671-4b5e-a23e-73e1b2a01889 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:41.736461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4bed2-5671-4b5e-a23e-73e1b2a01889 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:41.738139 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d4bed2-5671-4b5e-a23e-73e1b2a01889 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:41.742641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d4bed2-5671-4b5e-a23e-73e1b2a01889 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:41.757895 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9988205c-4656-465a-9616-8fa8b0303d51 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:41.802926 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba49618-dfbb-4916-a7ef-9354434be7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:41.808492 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba49618-dfbb-4916-a7ef-9354434be7cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:41.811350 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba49618-dfbb-4916-a7ef-9354434be7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:41.813777 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba49618-dfbb-4916-a7ef-9354434be7cf admin admin] PATH_INFO: `/v3/projects/f9c5816dc38b4a72a372862163cc2823` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:41.828200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba49618-dfbb-4916-a7ef-9354434be7cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9c5816dc38b4a72a372862163cc2823)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:41.828927 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 855/1690] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:51:41 2022] POST /identity/v3/users => generated 286 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:41.831061 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4bed2-5671-4b5e-a23e-73e1b2a01889 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:41.842632 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba49618-dfbb-4916-a7ef-9354434be7cf admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:41.895835 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4bed2-5671-4b5e-a23e-73e1b2a01889 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:41.913223 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e6b29-aada-4d0c-abbd-4efba34540ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:41.923020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e6b29-aada-4d0c-abbd-4efba34540ee admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:41.923646 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e6b29-aada-4d0c-abbd-4efba34540ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:41.924253 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e6b29-aada-4d0c-abbd-4efba34540ee admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:41.927871 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e6b29-aada-4d0c-abbd-4efba34540ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:41.931273 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e6b29-aada-4d0c-abbd-4efba34540ee admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:41.975858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 856/1691] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:41.982032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e6b29-aada-4d0c-abbd-4efba34540ee admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:42.026175 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e6b29-aada-4d0c-abbd-4efba34540ee admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:42.037618 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a921bb5-efeb-4d7c-9a42-283ea4741598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:42.039145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a921bb5-efeb-4d7c-9a42-283ea4741598 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:42.039704 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a921bb5-efeb-4d7c-9a42-283ea4741598 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:42.040307 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a921bb5-efeb-4d7c-9a42-283ea4741598 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:42.042322 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a921bb5-efeb-4d7c-9a42-283ea4741598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:42.044381 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a921bb5-efeb-4d7c-9a42-283ea4741598 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:42.064167 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 857/1692] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:42.066721 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a921bb5-efeb-4d7c-9a42-283ea4741598 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:42.106557 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a921bb5-efeb-4d7c-9a42-283ea4741598 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:42.108090 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fba49618-dfbb-4916-a7ef-9354434be7cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9c5816dc38b4a72a372862163cc2823', 'request_id': 'req-fba49618-dfbb-4916-a7ef-9354434be7cf'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:42.136398 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fba49618-dfbb-4916-a7ef-9354434be7cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9c5816dc38b4a72a372862163cc2823', 'request_id': 'req-fba49618-dfbb-4916-a7ef-9354434be7cf'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:42.162290 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fba49618-dfbb-4916-a7ef-9354434be7cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9c5816dc38b4a72a372862163cc2823', 'request_id': 'req-fba49618-dfbb-4916-a7ef-9354434be7cf'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:42.223154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fba49618-dfbb-4916-a7ef-9354434be7cf admin admin] The token cache is being invalidate because project f9c5816dc38b4a72a372862163cc2823 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:51:42.223827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fba49618-dfbb-4916-a7ef-9354434be7cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:42.226996 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fba49618-dfbb-4916-a7ef-9354434be7cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:42.231846 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 836/1693] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:51:41 2022] DELETE /identity/v3/projects/f9c5816dc38b4a72a372862163cc2823 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:51:42.238598 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba49618-dfbb-4916-a7ef-9354434be7cf admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:42.240550 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-fba49618-dfbb-4916-a7ef-9354434be7cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:42.313280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:42.375125 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d546b-eef6-4c96-9237-f5823755b151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:42.376702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d546b-eef6-4c96-9237-f5823755b151 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:42.376702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d546b-eef6-4c96-9237-f5823755b151 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:42.377330 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d546b-eef6-4c96-9237-f5823755b151 admin admin] PATH_INFO: `/v3/projects/e910ef13550e46d186f53e053846f923/users/6b17dba34af14a07a6383a3db75615fc/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:42.447201 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254d546b-eef6-4c96-9237-f5823755b151 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e910ef13550e46d186f53e053846f923, user_id=6b17dba34af14a07a6383a3db75615fc, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:42.455767 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254d546b-eef6-4c96-9237-f5823755b151 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:42.504213 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 858/1694] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:51:42 2022] PUT /identity/v3/projects/e910ef13550e46d186f53e053846f923/users/6b17dba34af14a07a6383a3db75615fc/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:42.514913 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba49618-dfbb-4916-a7ef-9354434be7cf admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:42.517688 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d29514-aee4-4c3a-85fb-2fecd229f2e0 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:42.518066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d29514-aee4-4c3a-85fb-2fecd229f2e0 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:42.518269 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d29514-aee4-4c3a-85fb-2fecd229f2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:42.545792 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:42.596687 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-53d29514-aee4-4c3a-85fb-2fecd229f2e0 None None] MFA Rules not processed for user `6b17dba34af14a07a6383a3db75615fc`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:42.602797 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-53d29514-aee4-4c3a-85fb-2fecd229f2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:42.778954 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab64d5d-7da6-414c-8a12-b73648fd58cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:42.779862 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab64d5d-7da6-414c-8a12-b73648fd58cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:42.780142 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab64d5d-7da6-414c-8a12-b73648fd58cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:42.780446 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab64d5d-7da6-414c-8a12-b73648fd58cc admin admin] PATH_INFO: `/v3/users/edbbf54a8f7e42f2b259c4c804b2d546` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:42.782031 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab64d5d-7da6-414c-8a12-b73648fd58cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=edbbf54a8f7e42f2b259c4c804b2d546)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:42.784213 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab64d5d-7da6-414c-8a12-b73648fd58cc admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:42.874552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 859/1695] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:42.878024 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d29514-aee4-4c3a-85fb-2fecd229f2e0 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:42.879968 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-53d29514-aee4-4c3a-85fb-2fecd229f2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:42.912743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-9ab64d5d-7da6-414c-8a12-b73648fd58cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edbbf54a8f7e42f2b259c4c804b2d546'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:42.933165 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d29514-aee4-4c3a-85fb-2fecd229f2e0 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:42.935700 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-9ab64d5d-7da6-414c-8a12-b73648fd58cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edbbf54a8f7e42f2b259c4c804b2d546'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:42.946414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f2326-8e55-4c87-8ee1-85e03bbd2410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:42.947297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f2326-8e55-4c87-8ee1-85e03bbd2410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:42.947695 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f2326-8e55-4c87-8ee1-85e03bbd2410 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:42.947971 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f2326-8e55-4c87-8ee1-85e03bbd2410 admin admin] PATH_INFO: `/v3/projects/80960e64cbd74e20a2932f80aa823ff5` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:42.951273 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f2326-8e55-4c87-8ee1-85e03bbd2410 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80960e64cbd74e20a2932f80aa823ff5)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:42.957356 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-9ab64d5d-7da6-414c-8a12-b73648fd58cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edbbf54a8f7e42f2b259c4c804b2d546'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:42.958243 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-9ab64d5d-7da6-414c-8a12-b73648fd58cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edbbf54a8f7e42f2b259c4c804b2d546'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:42.960351 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-9ab64d5d-7da6-414c-8a12-b73648fd58cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edbbf54a8f7e42f2b259c4c804b2d546'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:42.975994 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f2326-8e55-4c87-8ee1-85e03bbd2410 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:43.068969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 837/1696] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:51:42 2022] DELETE /identity/v3/users/edbbf54a8f7e42f2b259c4c804b2d546 => generated 0 bytes in 828 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:43.077394 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab64d5d-7da6-414c-8a12-b73648fd58cc admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:43.082837 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9ab64d5d-7da6-414c-8a12-b73648fd58cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:43.172723 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-087f2326-8e55-4c87-8ee1-85e03bbd2410 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80960e64cbd74e20a2932f80aa823ff5', 'request_id': 'req-087f2326-8e55-4c87-8ee1-85e03bbd2410'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:43.199882 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-087f2326-8e55-4c87-8ee1-85e03bbd2410 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80960e64cbd74e20a2932f80aa823ff5', 'request_id': 'req-087f2326-8e55-4c87-8ee1-85e03bbd2410'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:43.217635 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-087f2326-8e55-4c87-8ee1-85e03bbd2410 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80960e64cbd74e20a2932f80aa823ff5', 'request_id': 'req-087f2326-8e55-4c87-8ee1-85e03bbd2410'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:43.266486 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-087f2326-8e55-4c87-8ee1-85e03bbd2410 admin admin] The token cache is being invalidate because project 80960e64cbd74e20a2932f80aa823ff5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:51:43.267603 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-087f2326-8e55-4c87-8ee1-85e03bbd2410 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:43.268332 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-087f2326-8e55-4c87-8ee1-85e03bbd2410 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:43.272197 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 860/1697] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:51:42 2022] DELETE /identity/v3/projects/80960e64cbd74e20a2932f80aa823ff5 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:43.285096 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f2326-8e55-4c87-8ee1-85e03bbd2410 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:43.289347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-087f2326-8e55-4c87-8ee1-85e03bbd2410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:43.411118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:43.442935 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab64d5d-7da6-414c-8a12-b73648fd58cc admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:43.461085 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947905c-cc17-43f7-a9d6-069d005ed6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:43.461741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947905c-cc17-43f7-a9d6-069d005ed6e4 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:43.462077 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947905c-cc17-43f7-a9d6-069d005ed6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:43.462344 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947905c-cc17-43f7-a9d6-069d005ed6e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:43.463098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947905c-cc17-43f7-a9d6-069d005ed6e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:43.465404 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947905c-cc17-43f7-a9d6-069d005ed6e4 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:43.488548 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f2326-8e55-4c87-8ee1-85e03bbd2410 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:43.511617 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9272d6f-1681-4bda-9add-59c200efb771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:43.513660 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9272d6f-1681-4bda-9add-59c200efb771 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:43.515261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9272d6f-1681-4bda-9add-59c200efb771 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:43.516420 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9272d6f-1681-4bda-9add-59c200efb771 admin admin] PATH_INFO: `/v3/users/103038700e26414eb766e9d6eb018c69` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:43.520396 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9272d6f-1681-4bda-9add-59c200efb771 admin admin] RBAC: Authorizing `identity:delete_user(user_id=103038700e26414eb766e9d6eb018c69)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:43.528389 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9272d6f-1681-4bda-9add-59c200efb771 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:43.584111 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 838/1698] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:51:43 2022] POST /identity/v3/projects => generated 373 bytes in 514 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:43.589439 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947905c-cc17-43f7-a9d6-069d005ed6e4 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:43.597632 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b947905c-cc17-43f7-a9d6-069d005ed6e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:43.664549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:43.691268 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a9272d6f-1681-4bda-9add-59c200efb771 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '103038700e26414eb766e9d6eb018c69'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:43.755192 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a9272d6f-1681-4bda-9add-59c200efb771 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '103038700e26414eb766e9d6eb018c69'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:43.781339 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a9272d6f-1681-4bda-9add-59c200efb771 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '103038700e26414eb766e9d6eb018c69'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:43.805971 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a9272d6f-1681-4bda-9add-59c200efb771 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '103038700e26414eb766e9d6eb018c69'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:43.808583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a9272d6f-1681-4bda-9add-59c200efb771 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '103038700e26414eb766e9d6eb018c69'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:43.832191 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 861/1699] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:51:43 2022] DELETE /identity/v3/users/103038700e26414eb766e9d6eb018c69 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:43.846772 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947905c-cc17-43f7-a9d6-069d005ed6e4 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:43.970562 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0530045-49bc-41d1-b5b9-23a7e89673a4 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:43.971922 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0530045-49bc-41d1-b5b9-23a7e89673a4 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:43.972714 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0530045-49bc-41d1-b5b9-23a7e89673a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:44.044980 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf0cf1-3ba9-4991-84c0-93a9a1640855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:44.045711 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf0cf1-3ba9-4991-84c0-93a9a1640855 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:44.046206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf0cf1-3ba9-4991-84c0-93a9a1640855 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:44.046721 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf0cf1-3ba9-4991-84c0-93a9a1640855 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:44.048035 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaf0cf1-3ba9-4991-84c0-93a9a1640855 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:44.049438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-a0530045-49bc-41d1-b5b9-23a7e89673a4 None None] MFA Rules not processed for user `b6f722499abc409a8a345bcb70114eaf`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:44.050951 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaf0cf1-3ba9-4991-84c0-93a9a1640855 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:44.055284 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a0530045-49bc-41d1-b5b9-23a7e89673a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:44.138016 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 839/1700] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:51:43 2022] POST /identity/v3/users => generated 293 bytes in 545 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:44.140225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf0cf1-3ba9-4991-84c0-93a9a1640855 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:44.145606 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0aaf0cf1-3ba9-4991-84c0-93a9a1640855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:44.215741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf0cf1-3ba9-4991-84c0-93a9a1640855 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:44.239267 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323be68-dd97-4fcf-94c8-cfbab4d15177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:44.242545 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323be68-dd97-4fcf-94c8-cfbab4d15177 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:44.243918 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323be68-dd97-4fcf-94c8-cfbab4d15177 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:44.244446 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323be68-dd97-4fcf-94c8-cfbab4d15177 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:44.245754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3323be68-dd97-4fcf-94c8-cfbab4d15177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:44.255663 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3323be68-dd97-4fcf-94c8-cfbab4d15177 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:44.289693 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 840/1701] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 08:51:44.292807 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323be68-dd97-4fcf-94c8-cfbab4d15177 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:44.317979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323be68-dd97-4fcf-94c8-cfbab4d15177 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:44.330369 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec773a9-27e1-42a1-8b1f-6871bded5367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:44.331910 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec773a9-27e1-42a1-8b1f-6871bded5367 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:44.332715 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec773a9-27e1-42a1-8b1f-6871bded5367 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:44.333553 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec773a9-27e1-42a1-8b1f-6871bded5367 admin admin] PATH_INFO: `/v3/projects/ae6f3435f3e24ff3a5092d863daadbe8/users/79e7071a06f446b38f0c5a67358b99bd/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:44.351397 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 862/1702] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:44.379905 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0530045-49bc-41d1-b5b9-23a7e89673a4 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:44.382942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a0530045-49bc-41d1-b5b9-23a7e89673a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:44.401836 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec773a9-27e1-42a1-8b1f-6871bded5367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae6f3435f3e24ff3a5092d863daadbe8, user_id=79e7071a06f446b38f0c5a67358b99bd, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:44.408578 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec773a9-27e1-42a1-8b1f-6871bded5367 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:44.458784 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 841/1703] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:51:44 2022] PUT /identity/v3/projects/ae6f3435f3e24ff3a5092d863daadbe8/users/79e7071a06f446b38f0c5a67358b99bd/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:44.460731 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec773a9-27e1-42a1-8b1f-6871bded5367 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:44.609105 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:44.652837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0530045-49bc-41d1-b5b9-23a7e89673a4 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:44.726167 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec773a9-27e1-42a1-8b1f-6871bded5367 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:44.742397 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553fb35c-fd98-4909-bd84-866d1efaa7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:44.743548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fb35c-fd98-4909-bd84-866d1efaa7cb admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:44.744007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fb35c-fd98-4909-bd84-866d1efaa7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:44.745302 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fb35c-fd98-4909-bd84-866d1efaa7cb admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:44.746090 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553fb35c-fd98-4909-bd84-866d1efaa7cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:44.750564 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553fb35c-fd98-4909-bd84-866d1efaa7cb admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:44.772683 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 842/1704] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:44.775068 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553fb35c-fd98-4909-bd84-866d1efaa7cb admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:44.810489 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553fb35c-fd98-4909-bd84-866d1efaa7cb admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:44.819442 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24968119-b9af-49f0-b2fd-ea8e2b31e938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:44.820824 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24968119-b9af-49f0-b2fd-ea8e2b31e938 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:44.821378 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24968119-b9af-49f0-b2fd-ea8e2b31e938 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:44.822007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24968119-b9af-49f0-b2fd-ea8e2b31e938 admin admin] PATH_INFO: `/v3/projects/ae6f3435f3e24ff3a5092d863daadbe8/users/79e7071a06f446b38f0c5a67358b99bd/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:44.825883 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24968119-b9af-49f0-b2fd-ea8e2b31e938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae6f3435f3e24ff3a5092d863daadbe8, user_id=79e7071a06f446b38f0c5a67358b99bd, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:44.833038 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24968119-b9af-49f0-b2fd-ea8e2b31e938 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:44.846469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726a14f-2709-4835-96f6-0ec2c46d967e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:44.846469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726a14f-2709-4835-96f6-0ec2c46d967e service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:44.846469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726a14f-2709-4835-96f6-0ec2c46d967e service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:44.846469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726a14f-2709-4835-96f6-0ec2c46d967e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:44.852434 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:44.882688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 843/1705] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:51:44 2022] PUT /identity/v3/projects/ae6f3435f3e24ff3a5092d863daadbe8/users/79e7071a06f446b38f0c5a67358b99bd/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:44.918991 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375e951-1763-4ab8-8661-0a6321aac810 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:44.918991 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375e951-1763-4ab8-8661-0a6321aac810 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:44.918991 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375e951-1763-4ab8-8661-0a6321aac810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:44.989843 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-6375e951-1763-4ab8-8661-0a6321aac810 None None] MFA Rules not processed for user `79e7071a06f446b38f0c5a67358b99bd`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:44.996640 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6375e951-1763-4ab8-8661-0a6321aac810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:45.102402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726a14f-2709-4835-96f6-0ec2c46d967e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:45.104896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726a14f-2709-4835-96f6-0ec2c46d967e service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:45.144296 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 863/1706] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 765 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:51:45.318092 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 844/1707] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:45.337763 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726a14f-2709-4835-96f6-0ec2c46d967e service nova] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:45.339537 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-7726a14f-2709-4835-96f6-0ec2c46d967e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:45.557289 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726a14f-2709-4835-96f6-0ec2c46d967e service nova] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:45.680884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:45.709283 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc212b3-cd81-425e-9f48-40863225a2b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:45.710534 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc212b3-cd81-425e-9f48-40863225a2b6 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:45.710795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc212b3-cd81-425e-9f48-40863225a2b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:45.711417 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc212b3-cd81-425e-9f48-40863225a2b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:45.754975 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc212b3-cd81-425e-9f48-40863225a2b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:45.757209 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc212b3-cd81-425e-9f48-40863225a2b6 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:45.788589 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 864/1708] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:51:46.873332 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375e951-1763-4ab8-8661-0a6321aac810 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:46.937200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc212b3-cd81-425e-9f48-40863225a2b6 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:46.938751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-acc212b3-cd81-425e-9f48-40863225a2b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:47.087416 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375e951-1763-4ab8-8661-0a6321aac810 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:47.104626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df782b-22ad-47d3-97b8-8676b6943153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:47.105525 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df782b-22ad-47d3-97b8-8676b6943153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:47.105906 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df782b-22ad-47d3-97b8-8676b6943153 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:47.106285 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df782b-22ad-47d3-97b8-8676b6943153 admin admin] PATH_INFO: `/v3/users/6b17dba34af14a07a6383a3db75615fc` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:47.107751 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47df782b-22ad-47d3-97b8-8676b6943153 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b17dba34af14a07a6383a3db75615fc)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:47.110098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47df782b-22ad-47d3-97b8-8676b6943153 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:47.162217 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc212b3-cd81-425e-9f48-40863225a2b6 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:47.179816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26195c8-6add-4b36-8481-48e9a8425e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:47.180545 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26195c8-6add-4b36-8481-48e9a8425e49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:47.180882 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26195c8-6add-4b36-8481-48e9a8425e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:47.181211 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26195c8-6add-4b36-8481-48e9a8425e49 admin admin] PATH_INFO: `/v3/projects/533dafe3e1f84272a0de5027c880a5d9` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:47.183695 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26195c8-6add-4b36-8481-48e9a8425e49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=533dafe3e1f84272a0de5027c880a5d9)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:47.198840 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26195c8-6add-4b36-8481-48e9a8425e49 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:47.327610 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-47df782b-22ad-47d3-97b8-8676b6943153 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b17dba34af14a07a6383a3db75615fc'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:47.376143 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-47df782b-22ad-47d3-97b8-8676b6943153 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b17dba34af14a07a6383a3db75615fc'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:47.377815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a26195c8-6add-4b36-8481-48e9a8425e49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '533dafe3e1f84272a0de5027c880a5d9', 'request_id': 'req-a26195c8-6add-4b36-8481-48e9a8425e49'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:47.402050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a26195c8-6add-4b36-8481-48e9a8425e49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '533dafe3e1f84272a0de5027c880a5d9', 'request_id': 'req-a26195c8-6add-4b36-8481-48e9a8425e49'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:47.418945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a26195c8-6add-4b36-8481-48e9a8425e49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '533dafe3e1f84272a0de5027c880a5d9', 'request_id': 'req-a26195c8-6add-4b36-8481-48e9a8425e49'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:47.420041 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-47df782b-22ad-47d3-97b8-8676b6943153 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b17dba34af14a07a6383a3db75615fc'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:47.421916 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-47df782b-22ad-47d3-97b8-8676b6943153 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b17dba34af14a07a6383a3db75615fc'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:47.424002 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-47df782b-22ad-47d3-97b8-8676b6943153 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b17dba34af14a07a6383a3db75615fc'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:47.465585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 845/1709] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:51:46 2022] DELETE /identity/v3/users/6b17dba34af14a07a6383a3db75615fc => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:47.468022 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33518924-6daa-4ac2-85ae-7d3ac966b86f None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:47.468372 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33518924-6daa-4ac2-85ae-7d3ac966b86f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:47.469698 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33518924-6daa-4ac2-85ae-7d3ac966b86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:47.489640 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a26195c8-6add-4b36-8481-48e9a8425e49 admin admin] The token cache is being invalidate because project 533dafe3e1f84272a0de5027c880a5d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:51:47.497708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a26195c8-6add-4b36-8481-48e9a8425e49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:47.498012 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:47.500287 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a26195c8-6add-4b36-8481-48e9a8425e49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:47.512608 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 865/1710] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:51:46 2022] DELETE /identity/v3/projects/533dafe3e1f84272a0de5027c880a5d9 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:47.520040 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae19ad-9b67-4db2-89df-00855d4439b7 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:47.520247 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae19ad-9b67-4db2-89df-00855d4439b7 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:47.520669 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae19ad-9b67-4db2-89df-00855d4439b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:47.533180 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-33518924-6daa-4ac2-85ae-7d3ac966b86f None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:47.536179 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-33518924-6daa-4ac2-85ae-7d3ac966b86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:47.645946 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-bcae19ad-9b67-4db2-89df-00855d4439b7 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:47.653787 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-bcae19ad-9b67-4db2-89df-00855d4439b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:47.678490 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 846/1711] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:47.680773 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 866/1712] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:51:47.692661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1473a-8018-4850-8c29-8ccc9934b0f9 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:47.693166 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1473a-8018-4850-8c29-8ccc9934b0f9 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:47.693589 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1473a-8018-4850-8c29-8ccc9934b0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:47.743603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-fab1473a-8018-4850-8c29-8ccc9934b0f9 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:47.744819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-fab1473a-8018-4850-8c29-8ccc9934b0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:47.791523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 847/1713] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:47.797622 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae19ad-9b67-4db2-89df-00855d4439b7 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:47.823668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1473a-8018-4850-8c29-8ccc9934b0f9 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:47.838651 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-fab1473a-8018-4850-8c29-8ccc9934b0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:47.842820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae19ad-9b67-4db2-89df-00855d4439b7 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:47.863897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3450e-190d-4555-a74f-513e00518539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:47.868314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3450e-190d-4555-a74f-513e00518539 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:47.876540 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3450e-190d-4555-a74f-513e00518539 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:47.876920 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3450e-190d-4555-a74f-513e00518539 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:47.877656 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd3450e-190d-4555-a74f-513e00518539 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:51:47.878499 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd3450e-190d-4555-a74f-513e00518539 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:47.880884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd3450e-190d-4555-a74f-513e00518539 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:47.975324 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 867/1714] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:51:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:51:47.978988 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3450e-190d-4555-a74f-513e00518539 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:48.004909 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3450e-190d-4555-a74f-513e00518539 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:48.037632 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15b6b8-f3a7-4118-80f7-8e83c72a5ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:48.039474 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15b6b8-f3a7-4118-80f7-8e83c72a5ce4 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:48.039474 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15b6b8-f3a7-4118-80f7-8e83c72a5ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:48.039698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15b6b8-f3a7-4118-80f7-8e83c72a5ce4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:48.043369 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15b6b8-f3a7-4118-80f7-8e83c72a5ce4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:48.048032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad15b6b8-f3a7-4118-80f7-8e83c72a5ce4 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:48.133694 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1473a-8018-4850-8c29-8ccc9934b0f9 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:48.168898 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c178cff-7ca5-43c0-92b9-30c23e35e4a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:48.174304 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c178cff-7ca5-43c0-92b9-30c23e35e4a9 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:48.175661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c178cff-7ca5-43c0-92b9-30c23e35e4a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:48.175661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c178cff-7ca5-43c0-92b9-30c23e35e4a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:48.182680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 868/1715] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:51:47 2022] POST /identity/v3/projects => generated 379 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:48.182680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15b6b8-f3a7-4118-80f7-8e83c72a5ce4 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:48.183075 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ad15b6b8-f3a7-4118-80f7-8e83c72a5ce4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:48.230604 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c178cff-7ca5-43c0-92b9-30c23e35e4a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:48.246904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c178cff-7ca5-43c0-92b9-30c23e35e4a9 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:48.289347 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 848/1716] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:51:48.290354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c178cff-7ca5-43c0-92b9-30c23e35e4a9 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:48.316379 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c178cff-7ca5-43c0-92b9-30c23e35e4a9 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:48.425114 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b87a3-847d-48d0-9844-b9c2c9c754f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:48.425994 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b87a3-847d-48d0-9844-b9c2c9c754f8 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:48.428771 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b87a3-847d-48d0-9844-b9c2c9c754f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:48.428985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b87a3-847d-48d0-9844-b9c2c9c754f8 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:48.430721 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:48.431926 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b87a3-847d-48d0-9844-b9c2c9c754f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:48.442077 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b87a3-847d-48d0-9844-b9c2c9c754f8 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:48.486669 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15b6b8-f3a7-4118-80f7-8e83c72a5ce4 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:48.501174 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531643a-03b2-4054-b484-f9b68ab5763f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:48.504086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531643a-03b2-4054-b484-f9b68ab5763f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:48.504725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531643a-03b2-4054-b484-f9b68ab5763f admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:48.505303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531643a-03b2-4054-b484-f9b68ab5763f admin admin] PATH_INFO: `/v3/projects/43fa81d1a12a47e7b85b4cc551e3b916` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:48.506878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531643a-03b2-4054-b484-f9b68ab5763f admin admin] RBAC: Authorizing `identity:delete_project(project_id=43fa81d1a12a47e7b85b4cc551e3b916)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:48.509302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531643a-03b2-4054-b484-f9b68ab5763f admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:48.524212 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 849/1717] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:51:48 2022] POST /identity/v3/users => generated 290 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:48.532669 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b87a3-847d-48d0-9844-b9c2c9c754f8 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:48.580396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b87a3-847d-48d0-9844-b9c2c9c754f8 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:48.593526 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49fe5c-45d5-4104-9910-e60736490acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:48.595309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49fe5c-45d5-4104-9910-e60736490acd admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:48.597034 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49fe5c-45d5-4104-9910-e60736490acd admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:48.597951 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49fe5c-45d5-4104-9910-e60736490acd admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:48.599064 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49fe5c-45d5-4104-9910-e60736490acd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:48.605507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49fe5c-45d5-4104-9910-e60736490acd admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:48.636285 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 850/1718] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:48.647474 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49fe5c-45d5-4104-9910-e60736490acd admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:48.671631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7531643a-03b2-4054-b484-f9b68ab5763f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43fa81d1a12a47e7b85b4cc551e3b916', 'request_id': 'req-7531643a-03b2-4054-b484-f9b68ab5763f'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:48.685026 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49fe5c-45d5-4104-9910-e60736490acd admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:48.697102 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7531643a-03b2-4054-b484-f9b68ab5763f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43fa81d1a12a47e7b85b4cc551e3b916', 'request_id': 'req-7531643a-03b2-4054-b484-f9b68ab5763f'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:48.726077 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7531643a-03b2-4054-b484-f9b68ab5763f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43fa81d1a12a47e7b85b4cc551e3b916', 'request_id': 'req-7531643a-03b2-4054-b484-f9b68ab5763f'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:48.757451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:48.772097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7531643a-03b2-4054-b484-f9b68ab5763f admin admin] The token cache is being invalidate because project 43fa81d1a12a47e7b85b4cc551e3b916 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:51:48.772516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7531643a-03b2-4054-b484-f9b68ab5763f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:48.774188 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7531643a-03b2-4054-b484-f9b68ab5763f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:48.780413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 869/1719] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:51:48 2022] DELETE /identity/v3/projects/43fa81d1a12a47e7b85b4cc551e3b916 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:48.787397 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ae1e9-bd21-4aae-bdd4-1d5042f1b469 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:48.787823 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ae1e9-bd21-4aae-bdd4-1d5042f1b469 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:48.788246 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ae1e9-bd21-4aae-bdd4-1d5042f1b469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:48.858032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-928ae1e9-bd21-4aae-bdd4-1d5042f1b469 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:48.860423 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-928ae1e9-bd21-4aae-bdd4-1d5042f1b469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:48.918582 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ada76-f0d6-4a44-82ef-c8a7fcdf7d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:48.919388 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ada76-f0d6-4a44-82ef-c8a7fcdf7d5e admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:48.919671 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ada76-f0d6-4a44-82ef-c8a7fcdf7d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:48.920000 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ada76-f0d6-4a44-82ef-c8a7fcdf7d5e admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:48.920766 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ada76-f0d6-4a44-82ef-c8a7fcdf7d5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:48.923097 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ada76-f0d6-4a44-82ef-c8a7fcdf7d5e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:48.964894 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 851/1720] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:48.969642 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ada76-f0d6-4a44-82ef-c8a7fcdf7d5e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:48.975187 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c06ada76-f0d6-4a44-82ef-c8a7fcdf7d5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:49.013019 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ada76-f0d6-4a44-82ef-c8a7fcdf7d5e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:49.031681 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776c07f-d477-4916-8917-0f883ca8aa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:49.035567 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776c07f-d477-4916-8917-0f883ca8aa89 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:49.035567 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776c07f-d477-4916-8917-0f883ca8aa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:49.035874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776c07f-d477-4916-8917-0f883ca8aa89 admin admin] PATH_INFO: `/v3/projects/e848b3909e6b43bfa5b7c8b8e344f1e1/users/eb62cd1f714c4b89937916b2179163db/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:49.057157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 870/1721] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:51:49.083114 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52ee4b-37aa-4f7f-a5d0-97d37bc86054 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:49.083114 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52ee4b-37aa-4f7f-a5d0-97d37bc86054 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:49.086950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52ee4b-37aa-4f7f-a5d0-97d37bc86054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:49.172196 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776c07f-d477-4916-8917-0f883ca8aa89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e848b3909e6b43bfa5b7c8b8e344f1e1, user_id=eb62cd1f714c4b89937916b2179163db, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:49.210172 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776c07f-d477-4916-8917-0f883ca8aa89 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:49.253370 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-9c52ee4b-37aa-4f7f-a5d0-97d37bc86054 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:49.254796 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-9c52ee4b-37aa-4f7f-a5d0-97d37bc86054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:49.288956 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 852/1722] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:51:48 2022] PUT /identity/v3/projects/e848b3909e6b43bfa5b7c8b8e344f1e1/users/eb62cd1f714c4b89937916b2179163db/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:49.319433 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdeed32-8041-4a7d-98ea-2688fb178bfa None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:49.321728 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdeed32-8041-4a7d-98ea-2688fb178bfa None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:49.323007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdeed32-8041-4a7d-98ea-2688fb178bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:49.465326 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-dcdeed32-8041-4a7d-98ea-2688fb178bfa None None] MFA Rules not processed for user `eb62cd1f714c4b89937916b2179163db`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:49.466596 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-dcdeed32-8041-4a7d-98ea-2688fb178bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:49.490850 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 871/1723] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:49.490850 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266f420-9597-48d8-b841-0401df3c07bd None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:49.494182 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266f420-9597-48d8-b841-0401df3c07bd None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:49.494513 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266f420-9597-48d8-b841-0401df3c07bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:49.548743 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:49.605167 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-4266f420-9597-48d8-b841-0401df3c07bd None None] MFA Rules not processed for user `d065c8a972df4b8e92996ce27e96b486`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:49.606615 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4266f420-9597-48d8-b841-0401df3c07bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:49.816538 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 872/1724] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:49.817436 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266f420-9597-48d8-b841-0401df3c07bd None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:49.852394 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 853/1725] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 538 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:49.854288 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fdde38-391b-43a5-8ee9-c6a8d6a2923d None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:49.862749 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266f420-9597-48d8-b841-0401df3c07bd None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:49.863675 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fdde38-391b-43a5-8ee9-c6a8d6a2923d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:49.870563 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fdde38-391b-43a5-8ee9-c6a8d6a2923d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:49.879230 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24603f19-69a3-48c0-a655-41e03e50ccf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:49.880044 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24603f19-69a3-48c0-a655-41e03e50ccf5 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:49.880635 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24603f19-69a3-48c0-a655-41e03e50ccf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:49.881917 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24603f19-69a3-48c0-a655-41e03e50ccf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:49.884434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24603f19-69a3-48c0-a655-41e03e50ccf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:51:49.886159 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24603f19-69a3-48c0-a655-41e03e50ccf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:49.892568 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24603f19-69a3-48c0-a655-41e03e50ccf5 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:49.937317 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 873/1726] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:51:49.940488 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24603f19-69a3-48c0-a655-41e03e50ccf5 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:49.979360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24603f19-69a3-48c0-a655-41e03e50ccf5 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:49.989359 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42397cdd-489c-4b2c-87ff-2a290ea18cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:49.990022 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42397cdd-489c-4b2c-87ff-2a290ea18cee admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:49.990658 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42397cdd-489c-4b2c-87ff-2a290ea18cee admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:49.990948 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42397cdd-489c-4b2c-87ff-2a290ea18cee admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:49.991612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42397cdd-489c-4b2c-87ff-2a290ea18cee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:49.996625 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-22fdde38-391b-43a5-8ee9-c6a8d6a2923d None None] MFA Rules not processed for user `d065c8a972df4b8e92996ce27e96b486`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:49.999471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42397cdd-489c-4b2c-87ff-2a290ea18cee admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:50.008501 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-22fdde38-391b-43a5-8ee9-c6a8d6a2923d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:50.059895 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 854/1727] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:50.062214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fdde38-391b-43a5-8ee9-c6a8d6a2923d None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:50.107309 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 874/1728] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:51:49 2022] POST /identity/v3/projects => generated 381 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:50.111215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42397cdd-489c-4b2c-87ff-2a290ea18cee admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:50.121373 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-42397cdd-489c-4b2c-87ff-2a290ea18cee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:50.139547 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fdde38-391b-43a5-8ee9-c6a8d6a2923d None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:50.343880 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44d012-a668-43aa-b827-88db4cbc6245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:50.347788 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44d012-a668-43aa-b827-88db4cbc6245 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:50.348437 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44d012-a668-43aa-b827-88db4cbc6245 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:50.349132 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44d012-a668-43aa-b827-88db4cbc6245 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:50.354516 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c44d012-a668-43aa-b827-88db4cbc6245 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:50.366326 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c44d012-a668-43aa-b827-88db4cbc6245 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:50.503134 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:50.512205 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42397cdd-489c-4b2c-87ff-2a290ea18cee admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:50.533809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a0e65-50bd-4229-bf5c-e4d1bee7a2e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:50.535653 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a0e65-50bd-4229-bf5c-e4d1bee7a2e1 service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:50.536097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a0e65-50bd-4229-bf5c-e4d1bee7a2e1 service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:50.536617 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 855/1729] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:51:50 2022] POST /identity/v3/projects => generated 367 bytes in 476 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:50.538115 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44d012-a668-43aa-b827-88db4cbc6245 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:50.540922 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a0e65-50bd-4229-bf5c-e4d1bee7a2e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:50.749771 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a0e65-50bd-4229-bf5c-e4d1bee7a2e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:50.752112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6a0e65-50bd-4229-bf5c-e4d1bee7a2e1 service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:50.757068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:50.771274 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 875/1730] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:51:50.785386 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a0e65-50bd-4229-bf5c-e4d1bee7a2e1 service nova] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:50.790062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44d012-a668-43aa-b827-88db4cbc6245 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:50.824907 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a0e65-50bd-4229-bf5c-e4d1bee7a2e1 service nova] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:50.830192 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06901d03-1322-4deb-85d6-d974ce236621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:50.831350 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06901d03-1322-4deb-85d6-d974ce236621 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:50.832141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06901d03-1322-4deb-85d6-d974ce236621 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:50.832655 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06901d03-1322-4deb-85d6-d974ce236621 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:50.833760 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06901d03-1322-4deb-85d6-d974ce236621 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:50.834649 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bf3cd-a1df-48c7-ae54-3e1a5c430eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:50.835468 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bf3cd-a1df-48c7-ae54-3e1a5c430eac admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:50.836084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bf3cd-a1df-48c7-ae54-3e1a5c430eac admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:50.836687 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bf3cd-a1df-48c7-ae54-3e1a5c430eac admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:50.837361 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06901d03-1322-4deb-85d6-d974ce236621 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:50.838022 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118bf3cd-a1df-48c7-ae54-3e1a5c430eac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:50.841645 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118bf3cd-a1df-48c7-ae54-3e1a5c430eac admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:50.929454 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 876/1731] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:51:50 2022] POST /identity/v3/users => generated 284 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:50.935346 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bf3cd-a1df-48c7-ae54-3e1a5c430eac admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:50.937250 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 856/1732] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:51:50 2022] POST /identity/v3/users => generated 297 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:50.948200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06901d03-1322-4deb-85d6-d974ce236621 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:50.958242 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bf3cd-a1df-48c7-ae54-3e1a5c430eac admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:50.988042 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e11fa5-79da-41ad-9982-f9d6beb5cdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:50.994087 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e11fa5-79da-41ad-9982-f9d6beb5cdd4 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:50.994574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e11fa5-79da-41ad-9982-f9d6beb5cdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:50.995173 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e11fa5-79da-41ad-9982-f9d6beb5cdd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:51.001110 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e11fa5-79da-41ad-9982-f9d6beb5cdd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:51.003089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e11fa5-79da-41ad-9982-f9d6beb5cdd4 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:51.005734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06901d03-1322-4deb-85d6-d974ce236621 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:51.021217 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 877/1733] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:51.041312 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e11fa5-79da-41ad-9982-f9d6beb5cdd4 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:51.052223 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff623255-f630-4f1b-a929-09f65fe7e1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:51.053721 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff623255-f630-4f1b-a929-09f65fe7e1dc admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:51.053868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff623255-f630-4f1b-a929-09f65fe7e1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:51.054249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff623255-f630-4f1b-a929-09f65fe7e1dc admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:51.055471 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff623255-f630-4f1b-a929-09f65fe7e1dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:51.058475 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff623255-f630-4f1b-a929-09f65fe7e1dc admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:51.065724 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e11fa5-79da-41ad-9982-f9d6beb5cdd4 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:51.073250 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa0566-cfed-4939-9313-3a8814cb98eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:51.073250 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa0566-cfed-4939-9313-3a8814cb98eb admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:51.073663 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa0566-cfed-4939-9313-3a8814cb98eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:51.074111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa0566-cfed-4939-9313-3a8814cb98eb admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:51.074900 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa0566-cfed-4939-9313-3a8814cb98eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:51.077172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa0566-cfed-4939-9313-3a8814cb98eb admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:51.102029 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 878/1734] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:51.107304 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa0566-cfed-4939-9313-3a8814cb98eb admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:51.119053 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 857/1735] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:51.120093 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff623255-f630-4f1b-a929-09f65fe7e1dc admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:51.157690 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff623255-f630-4f1b-a929-09f65fe7e1dc admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:51.171665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16887c-2878-4fc5-a749-bb776fabd182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:51.172514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16887c-2878-4fc5-a749-bb776fabd182 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:51.172984 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16887c-2878-4fc5-a749-bb776fabd182 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:51.173367 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16887c-2878-4fc5-a749-bb776fabd182 admin admin] PATH_INFO: `/v3/projects/e2593c59fb474db69cfce5a98d514c75/users/af2bb237205c4b3e80ed68541ff40972/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:51.183438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa0566-cfed-4939-9313-3a8814cb98eb admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:51.200060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62fc907-dd65-4be5-b09f-2d12b888ddca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:51.202452 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62fc907-dd65-4be5-b09f-2d12b888ddca admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:51.203325 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62fc907-dd65-4be5-b09f-2d12b888ddca admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:51.205783 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62fc907-dd65-4be5-b09f-2d12b888ddca admin admin] PATH_INFO: `/v3/projects/100c2c9e36264dd995b1597899de33aa/users/a733093a4907444899d8507117675b0b/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:51.227696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16887c-2878-4fc5-a749-bb776fabd182 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2593c59fb474db69cfce5a98d514c75, user_id=af2bb237205c4b3e80ed68541ff40972, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:51.233084 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16887c-2878-4fc5-a749-bb776fabd182 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:51.284507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 858/1736] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:51:51 2022] PUT /identity/v3/projects/e2593c59fb474db69cfce5a98d514c75/users/af2bb237205c4b3e80ed68541ff40972/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:51:51.290459 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62fc907-dd65-4be5-b09f-2d12b888ddca admin admin] RBAC: Authorizing `identity:create_grant(project_id=100c2c9e36264dd995b1597899de33aa, user_id=a733093a4907444899d8507117675b0b, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:51.296357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16887c-2878-4fc5-a749-bb776fabd182 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:51.298743 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62fc907-dd65-4be5-b09f-2d12b888ddca admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:51.348828 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 879/1737] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:51:51 2022] PUT /identity/v3/projects/100c2c9e36264dd995b1597899de33aa/users/a733093a4907444899d8507117675b0b/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:51:51.384690 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51460b71-86c1-4697-8068-f73be24fc305 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:51.384690 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51460b71-86c1-4697-8068-f73be24fc305 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:51.385018 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51460b71-86c1-4697-8068-f73be24fc305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:51.518065 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-51460b71-86c1-4697-8068-f73be24fc305 None None] MFA Rules not processed for user `a733093a4907444899d8507117675b0b`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:51.519853 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-51460b71-86c1-4697-8068-f73be24fc305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:51.603852 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16887c-2878-4fc5-a749-bb776fabd182 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:51.798441 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 880/1738] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:51.812179 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51460b71-86c1-4697-8068-f73be24fc305 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:51.862409 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228064bf-00f3-4db9-a72a-d01b4e8759cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:51.864544 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228064bf-00f3-4db9-a72a-d01b4e8759cf admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:51.864947 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228064bf-00f3-4db9-a72a-d01b4e8759cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:51.866027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228064bf-00f3-4db9-a72a-d01b4e8759cf admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:51.867083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228064bf-00f3-4db9-a72a-d01b4e8759cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:51.873237 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228064bf-00f3-4db9-a72a-d01b4e8759cf admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:51.881511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:51.926763 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 859/1739] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 631 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:51.928546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228064bf-00f3-4db9-a72a-d01b4e8759cf admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:51.974915 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228064bf-00f3-4db9-a72a-d01b4e8759cf admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:51.999810 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504fa46-8f6f-46c6-ab73-cf4a302a041d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:52.003414 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504fa46-8f6f-46c6-ab73-cf4a302a041d admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:52.007624 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504fa46-8f6f-46c6-ab73-cf4a302a041d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:52.007843 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504fa46-8f6f-46c6-ab73-cf4a302a041d admin admin] PATH_INFO: `/v3/projects/e2593c59fb474db69cfce5a98d514c75/users/af2bb237205c4b3e80ed68541ff40972/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:52.023133 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b504fa46-8f6f-46c6-ab73-cf4a302a041d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2593c59fb474db69cfce5a98d514c75, user_id=af2bb237205c4b3e80ed68541ff40972, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:52.027198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51460b71-86c1-4697-8068-f73be24fc305 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:52.033582 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e1ab2-ca5c-48ce-b90c-40f131ae46a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:52.034468 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e1ab2-ca5c-48ce-b90c-40f131ae46a5 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:52.035321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e1ab2-ca5c-48ce-b90c-40f131ae46a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:52.036970 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e1ab2-ca5c-48ce-b90c-40f131ae46a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:52.038261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e1ab2-ca5c-48ce-b90c-40f131ae46a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:52.038623 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b504fa46-8f6f-46c6-ab73-cf4a302a041d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:52.041272 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e1ab2-ca5c-48ce-b90c-40f131ae46a5 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:52.106419 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 860/1740] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:51:51 2022] PUT /identity/v3/projects/e2593c59fb474db69cfce5a98d514c75/users/af2bb237205c4b3e80ed68541ff40972/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:52.113520 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 881/1741] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:51:51 2022] POST /identity/v3/projects => generated 367 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:52.119174 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504fa46-8f6f-46c6-ab73-cf4a302a041d admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:52.128637 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999012c9-0e49-46fb-a5cf-c6096bac53df None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:52.129272 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999012c9-0e49-46fb-a5cf-c6096bac53df None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:52.129902 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999012c9-0e49-46fb-a5cf-c6096bac53df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:52.194515 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:52.222369 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-999012c9-0e49-46fb-a5cf-c6096bac53df None None] MFA Rules not processed for user `af2bb237205c4b3e80ed68541ff40972`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:52.224254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-999012c9-0e49-46fb-a5cf-c6096bac53df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:52.400135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504fa46-8f6f-46c6-ab73-cf4a302a041d admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:52.424788 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 882/1742] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:52 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:51:52.425939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999012c9-0e49-46fb-a5cf-c6096bac53df None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:52.430046 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-999012c9-0e49-46fb-a5cf-c6096bac53df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:52.621591 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14251ab4-347e-46ee-8584-0af4d8fcb393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:52.622185 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14251ab4-347e-46ee-8584-0af4d8fcb393 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:52.624649 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14251ab4-347e-46ee-8584-0af4d8fcb393 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:52.626015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14251ab4-347e-46ee-8584-0af4d8fcb393 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:52.628517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14251ab4-347e-46ee-8584-0af4d8fcb393 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:52.637286 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14251ab4-347e-46ee-8584-0af4d8fcb393 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:52.713305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 861/1743] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:51:52 2022] POST /identity/v3/users => generated 290 bytes in 598 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:52.715877 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14251ab4-347e-46ee-8584-0af4d8fcb393 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:52.724842 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-14251ab4-347e-46ee-8584-0af4d8fcb393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:52.824953 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999012c9-0e49-46fb-a5cf-c6096bac53df None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:52.842872 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b28db-4b78-4278-b52e-5223f4a67089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:52.843838 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b28db-4b78-4278-b52e-5223f4a67089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:52.844265 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b28db-4b78-4278-b52e-5223f4a67089 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:52.844655 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b28db-4b78-4278-b52e-5223f4a67089 admin admin] PATH_INFO: `/v3/projects/e910ef13550e46d186f53e053846f923` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:52.851198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b28db-4b78-4278-b52e-5223f4a67089 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e910ef13550e46d186f53e053846f923)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:52.858423 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b28db-4b78-4278-b52e-5223f4a67089 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:53.003356 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b47b28db-4b78-4278-b52e-5223f4a67089 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e910ef13550e46d186f53e053846f923', 'request_id': 'req-b47b28db-4b78-4278-b52e-5223f4a67089'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:53.019798 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b47b28db-4b78-4278-b52e-5223f4a67089 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e910ef13550e46d186f53e053846f923', 'request_id': 'req-b47b28db-4b78-4278-b52e-5223f4a67089'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:53.034363 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b47b28db-4b78-4278-b52e-5223f4a67089 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e910ef13550e46d186f53e053846f923', 'request_id': 'req-b47b28db-4b78-4278-b52e-5223f4a67089'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:53.080374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14251ab4-347e-46ee-8584-0af4d8fcb393 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:53.084888 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b47b28db-4b78-4278-b52e-5223f4a67089 admin admin] The token cache is being invalidate because project e910ef13550e46d186f53e053846f923 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:51:53.085866 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b47b28db-4b78-4278-b52e-5223f4a67089 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:53.087343 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b47b28db-4b78-4278-b52e-5223f4a67089 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:53.093654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 883/1744] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:51:52 2022] DELETE /identity/v3/projects/e910ef13550e46d186f53e053846f923 => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:53.098791 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ba874-0140-4045-9898-19cfcd7fb6dc None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:53.106573 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ba874-0140-4045-9898-19cfcd7fb6dc None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:53.106728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ba874-0140-4045-9898-19cfcd7fb6dc None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:53.108902 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 884/1745] 10.222.0.71 () {54 vars in 912 bytes} [Fri Aug 26 08:51:53 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:51:53.110051 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ba874-0140-4045-9898-19cfcd7fb6dc None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:53.118154 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2f3ba874-0140-4045-9898-19cfcd7fb6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:53.189906 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:53.203036 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:53.245252 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be3595-a495-40f1-86dc-8f9be4a0743e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:53.246646 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be3595-a495-40f1-86dc-8f9be4a0743e service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:53.247185 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be3595-a495-40f1-86dc-8f9be4a0743e service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:53.248209 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be3595-a495-40f1-86dc-8f9be4a0743e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:53.253796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-19be3595-a495-40f1-86dc-8f9be4a0743e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:53.294839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ba874-0140-4045-9898-19cfcd7fb6dc None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:53.311038 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839fb847-2af8-4b2a-b806-53b3a2f85ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:53.311936 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fb847-2af8-4b2a-b806-53b3a2f85ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:53.312353 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fb847-2af8-4b2a-b806-53b3a2f85ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:53.312903 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fb847-2af8-4b2a-b806-53b3a2f85ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:53.313708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839fb847-2af8-4b2a-b806-53b3a2f85ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:53.316303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839fb847-2af8-4b2a-b806-53b3a2f85ff7 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:53.347340 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 885/1746] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:53.347600 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbaf22-197e-41ab-bdad-f4933762dec5 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:53.347831 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbaf22-197e-41ab-bdad-f4933762dec5 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:53.348118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbaf22-197e-41ab-bdad-f4933762dec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:53.389884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-f9fbaf22-197e-41ab-bdad-f4933762dec5 None None] MFA Rules not processed for user `b6f722499abc409a8a345bcb70114eaf`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:53.391321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f9fbaf22-197e-41ab-bdad-f4933762dec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:53.421725 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be3595-a495-40f1-86dc-8f9be4a0743e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:53.429993 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19be3595-a495-40f1-86dc-8f9be4a0743e service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:53.460638 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 862/1747] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:52 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 746 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:51:53.461973 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be3595-a495-40f1-86dc-8f9be4a0743e service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:53.463717 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-19be3595-a495-40f1-86dc-8f9be4a0743e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:53.491066 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19be3595-a495-40f1-86dc-8f9be4a0743e service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:53.496380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a149fb1-fb30-48c7-93ac-cfab3e10e86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:53.497043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a149fb1-fb30-48c7-93ac-cfab3e10e86b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:53.497349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a149fb1-fb30-48c7-93ac-cfab3e10e86b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:53.497705 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a149fb1-fb30-48c7-93ac-cfab3e10e86b admin admin] PATH_INFO: `/v3/users/79e7071a06f446b38f0c5a67358b99bd` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:53.499026 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a149fb1-fb30-48c7-93ac-cfab3e10e86b admin admin] RBAC: Authorizing `identity:delete_user(user_id=79e7071a06f446b38f0c5a67358b99bd)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:53.501768 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a149fb1-fb30-48c7-93ac-cfab3e10e86b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:53.561463 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 886/1748] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:53.563714 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461081a-2597-47c1-83e8-32f0cd747249 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:53.564144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461081a-2597-47c1-83e8-32f0cd747249 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:53.564549 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1461081a-2597-47c1-83e8-32f0cd747249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:53.643398 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5a149fb1-fb30-48c7-93ac-cfab3e10e86b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79e7071a06f446b38f0c5a67358b99bd'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:53.660647 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5a149fb1-fb30-48c7-93ac-cfab3e10e86b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79e7071a06f446b38f0c5a67358b99bd'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:53.676580 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-1461081a-2597-47c1-83e8-32f0cd747249 None None] MFA Rules not processed for user `4aa8938a504c4c4f8815478e520482d8`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:53.678660 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5a149fb1-fb30-48c7-93ac-cfab3e10e86b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79e7071a06f446b38f0c5a67358b99bd'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:53.679978 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5a149fb1-fb30-48c7-93ac-cfab3e10e86b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79e7071a06f446b38f0c5a67358b99bd'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:53.681915 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5a149fb1-fb30-48c7-93ac-cfab3e10e86b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79e7071a06f446b38f0c5a67358b99bd'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:53.686207 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1461081a-2597-47c1-83e8-32f0cd747249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:53.761410 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 863/1749] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:51:53 2022] DELETE /identity/v3/users/79e7071a06f446b38f0c5a67358b99bd => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:53.762388 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a149fb1-fb30-48c7-93ac-cfab3e10e86b admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:53.763852 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5a149fb1-fb30-48c7-93ac-cfab3e10e86b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:53.928445 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 887/1750] 10.222.0.71 () {58 vars in 1009 bytes} [Fri Aug 26 08:51:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:54.010456 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a149fb1-fb30-48c7-93ac-cfab3e10e86b admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:54.027422 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77c9fb-8f34-4e8e-b4b0-e5c92e4318d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:54.028216 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77c9fb-8f34-4e8e-b4b0-e5c92e4318d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:54.028503 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77c9fb-8f34-4e8e-b4b0-e5c92e4318d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:54.028862 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77c9fb-8f34-4e8e-b4b0-e5c92e4318d1 admin admin] PATH_INFO: `/v3/projects/081e0916084e42a58615501194d6364e/users/96455864e2a3489dad2de8569e19c119/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:54.034205 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:54.074462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77c9fb-8f34-4e8e-b4b0-e5c92e4318d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=081e0916084e42a58615501194d6364e, user_id=96455864e2a3489dad2de8569e19c119, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:54.077448 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77c9fb-8f34-4e8e-b4b0-e5c92e4318d1 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:54.113726 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 864/1751] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:51:53 2022] PUT /identity/v3/projects/081e0916084e42a58615501194d6364e/users/96455864e2a3489dad2de8569e19c119/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:54.115714 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461081a-2597-47c1-83e8-32f0cd747249 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:54.168000 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1461081a-2597-47c1-83e8-32f0cd747249 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:54.168330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77c9fb-8f34-4e8e-b4b0-e5c92e4318d1 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:54.171482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-dd77c9fb-8f34-4e8e-b4b0-e5c92e4318d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:54.415444 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77c9fb-8f34-4e8e-b4b0-e5c92e4318d1 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:54.421233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ad7d4-c478-4740-b502-5d690f57b9b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:54.422633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ad7d4-c478-4740-b502-5d690f57b9b1 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:54.426058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ad7d4-c478-4740-b502-5d690f57b9b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:54.426759 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ad7d4-c478-4740-b502-5d690f57b9b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:54.441874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ad7d4-c478-4740-b502-5d690f57b9b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:54.446970 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ad7d4-c478-4740-b502-5d690f57b9b1 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:54.531550 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c11889-2b25-4775-bcad-679c8a063650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:54.532365 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c11889-2b25-4775-bcad-679c8a063650 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:54.532711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c11889-2b25-4775-bcad-679c8a063650 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:54.532914 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c11889-2b25-4775-bcad-679c8a063650 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:54.533527 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c11889-2b25-4775-bcad-679c8a063650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:54.534425 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 865/1752] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:51:54.538411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c11889-2b25-4775-bcad-679c8a063650 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:54.546786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:54.578998 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 888/1753] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 464 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:54.584444 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ad7d4-c478-4740-b502-5d690f57b9b1 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:54.632512 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c11889-2b25-4775-bcad-679c8a063650 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:54.634308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d9c11889-2b25-4775-bcad-679c8a063650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:54.643593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ad7d4-c478-4740-b502-5d690f57b9b1 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:54.703407 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19412618-a1fe-46c9-a953-b9f453917eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:54.704074 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19412618-a1fe-46c9-a953-b9f453917eb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:54.704518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19412618-a1fe-46c9-a953-b9f453917eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:54.704979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19412618-a1fe-46c9-a953-b9f453917eb2 admin admin] PATH_INFO: `/v3/projects/081e0916084e42a58615501194d6364e/users/96455864e2a3489dad2de8569e19c119/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:54.709748 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19412618-a1fe-46c9-a953-b9f453917eb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=081e0916084e42a58615501194d6364e, user_id=96455864e2a3489dad2de8569e19c119, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:54.712896 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19412618-a1fe-46c9-a953-b9f453917eb2 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:54.743164 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 866/1754] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:51:54 2022] PUT /identity/v3/projects/081e0916084e42a58615501194d6364e/users/96455864e2a3489dad2de8569e19c119/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:54.745320 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c84c29-e72d-4714-b135-a1066b4ff35c None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:54.745694 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c84c29-e72d-4714-b135-a1066b4ff35c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:54.746379 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c84c29-e72d-4714-b135-a1066b4ff35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:54.853252 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c11889-2b25-4775-bcad-679c8a063650 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:54.889314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-65c84c29-e72d-4714-b135-a1066b4ff35c None None] MFA Rules not processed for user `63e1c5a24be04ca4abd5d5109c5763f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:54.897197 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-65c84c29-e72d-4714-b135-a1066b4ff35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:55.100340 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8beca8-c4dc-4d12-b7a1-d2e18e4d5567 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af2bb237205c4b3e80ed68541ff40972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2593c59fb474db69cfce5a98d514c75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:55.101109 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8beca8-c4dc-4d12-b7a1-d2e18e4d5567 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:55.102600 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8beca8-c4dc-4d12-b7a1-d2e18e4d5567 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:55.102908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8beca8-c4dc-4d12-b7a1-d2e18e4d5567 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:55.106601 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8beca8-c4dc-4d12-b7a1-d2e18e4d5567 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:51:55.106969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8beca8-c4dc-4d12-b7a1-d2e18e4d5567 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:55.108988 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8beca8-c4dc-4d12-b7a1-d2e18e4d5567 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:55.174970 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 867/1755] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:55.174970 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99e5be-3624-4d8a-8801-0e630c276548 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:55.174970 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99e5be-3624-4d8a-8801-0e630c276548 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:55.174970 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99e5be-3624-4d8a-8801-0e630c276548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:55.177908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 889/1756] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 546 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:51:55.183418 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8beca8-c4dc-4d12-b7a1-d2e18e4d5567 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:55.203303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8beca8-c4dc-4d12-b7a1-d2e18e4d5567 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:55.211561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731b270-a973-45a3-ab58-98ffc0550776 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af2bb237205c4b3e80ed68541ff40972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2593c59fb474db69cfce5a98d514c75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:55.213025 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731b270-a973-45a3-ab58-98ffc0550776 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:55.213511 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731b270-a973-45a3-ab58-98ffc0550776 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:55.213985 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731b270-a973-45a3-ab58-98ffc0550776 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:55.215844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731b270-a973-45a3-ab58-98ffc0550776 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:55.221204 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731b270-a973-45a3-ab58-98ffc0550776 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:55.322691 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-7f99e5be-3624-4d8a-8801-0e630c276548 None None] MFA Rules not processed for user `96455864e2a3489dad2de8569e19c119`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:55.331616 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 890/1757] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:51:55 2022] POST /identity/v3/projects => generated 352 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:55.333077 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731b270-a973-45a3-ab58-98ffc0550776 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:55.334099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7f99e5be-3624-4d8a-8801-0e630c276548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:55.654364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:55.741941 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731b270-a973-45a3-ab58-98ffc0550776 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:55.760347 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 868/1758] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 597 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:55.763668 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8081e5b-e0f8-44ee-a109-b97678984e5f tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af2bb237205c4b3e80ed68541ff40972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2593c59fb474db69cfce5a98d514c75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:55.767762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8081e5b-e0f8-44ee-a109-b97678984e5f tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:55.768634 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8081e5b-e0f8-44ee-a109-b97678984e5f tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:55.775636 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8081e5b-e0f8-44ee-a109-b97678984e5f tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:55.781629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8081e5b-e0f8-44ee-a109-b97678984e5f tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:51:55.782618 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8081e5b-e0f8-44ee-a109-b97678984e5f tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:55.789413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8081e5b-e0f8-44ee-a109-b97678984e5f tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:55.792557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f99e5be-3624-4d8a-8801-0e630c276548 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:55.801017 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7f99e5be-3624-4d8a-8801-0e630c276548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:55.875742 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 891/1759] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 544 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:51:56.008794 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8081e5b-e0f8-44ee-a109-b97678984e5f tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:56.043594 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8081e5b-e0f8-44ee-a109-b97678984e5f tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:56.074330 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eabd01-d9b0-4e34-9238-5b15922fc354 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af2bb237205c4b3e80ed68541ff40972', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e2593c59fb474db69cfce5a98d514c75', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:56.078139 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eabd01-d9b0-4e34-9238-5b15922fc354 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:56.080341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eabd01-d9b0-4e34-9238-5b15922fc354 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:56.081902 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eabd01-d9b0-4e34-9238-5b15922fc354 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] PATH_INFO: `/v3/projects/13aa761bf24a453289af959ea1970008` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:56.088494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eabd01-d9b0-4e34-9238-5b15922fc354 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] RBAC: Authorizing `identity:delete_project(project_id=13aa761bf24a453289af959ea1970008)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:56.105559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eabd01-d9b0-4e34-9238-5b15922fc354 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:56.135643 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f99e5be-3624-4d8a-8801-0e630c276548 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:56.142278 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:56.149605 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2213ef-694c-4406-b382-0ffe9e48c2af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:56.151923 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2213ef-694c-4406-b382-0ffe9e48c2af service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:56.152165 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2213ef-694c-4406-b382-0ffe9e48c2af service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:56.154878 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2213ef-694c-4406-b382-0ffe9e48c2af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:56.194139 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2213ef-694c-4406-b382-0ffe9e48c2af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:56.197482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2213ef-694c-4406-b382-0ffe9e48c2af service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:56.249210 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 869/1760] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:51:56.313735 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-56eabd01-d9b0-4e34-9238-5b15922fc354 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13aa761bf24a453289af959ea1970008', 'request_id': 'req-56eabd01-d9b0-4e34-9238-5b15922fc354'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:56.331818 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-56eabd01-d9b0-4e34-9238-5b15922fc354 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13aa761bf24a453289af959ea1970008', 'request_id': 'req-56eabd01-d9b0-4e34-9238-5b15922fc354'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:56.350843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-56eabd01-d9b0-4e34-9238-5b15922fc354 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13aa761bf24a453289af959ea1970008', 'request_id': 'req-56eabd01-d9b0-4e34-9238-5b15922fc354'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:56.437430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-56eabd01-d9b0-4e34-9238-5b15922fc354 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] The token cache is being invalidate because project 13aa761bf24a453289af959ea1970008 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:51:56.438026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-56eabd01-d9b0-4e34-9238-5b15922fc354 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:56.439927 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-56eabd01-d9b0-4e34-9238-5b15922fc354 tempest-VolumeQuotaClassesTest-1590497614 tempest-VolumeQuotaClassesTest-1590497614-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:56.450429 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 892/1761] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:51:56 2022] DELETE /identity/v3/projects/13aa761bf24a453289af959ea1970008 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:51:56.512843 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2213ef-694c-4406-b382-0ffe9e48c2af service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:56.516266 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8e2213ef-694c-4406-b382-0ffe9e48c2af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:56.548624 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2213ef-694c-4406-b382-0ffe9e48c2af service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:56.693778 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043b6da-f153-4222-964a-3c9203cfacb3 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:56.694485 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043b6da-f153-4222-964a-3c9203cfacb3 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:56.695015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043b6da-f153-4222-964a-3c9203cfacb3 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:56.697736 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 893/1762] 10.222.0.71 () {54 vars in 912 bytes} [Fri Aug 26 08:51:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:51:56.728370 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4c75b-b91b-4b90-aba9-54f363b97205 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:56.729413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4c75b-b91b-4b90-aba9-54f363b97205 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:56.730875 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4c75b-b91b-4b90-aba9-54f363b97205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:56.840762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e4b4c75b-b91b-4b90-aba9-54f363b97205 None None] MFA Rules not processed for user `4aa8938a504c4c4f8815478e520482d8`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:56.852614 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e4b4c75b-b91b-4b90-aba9-54f363b97205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:56.866114 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c54a09-e1f3-48b2-b87e-33305bcae09b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:56.869039 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c54a09-e1f3-48b2-b87e-33305bcae09b service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:56.873060 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c54a09-e1f3-48b2-b87e-33305bcae09b service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:56.873791 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c54a09-e1f3-48b2-b87e-33305bcae09b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:56.880466 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c4c54a09-e1f3-48b2-b87e-33305bcae09b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:57.090685 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c54a09-e1f3-48b2-b87e-33305bcae09b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:57.092249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c54a09-e1f3-48b2-b87e-33305bcae09b service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:57.103478 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 894/1763] 10.222.0.71 () {58 vars in 1009 bytes} [Fri Aug 26 08:51:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:57.104158 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 870/1764] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:51:57.104915 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b4c75b-b91b-4b90-aba9-54f363b97205 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:57.105161 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c54a09-e1f3-48b2-b87e-33305bcae09b service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:57.106830 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c4c54a09-e1f3-48b2-b87e-33305bcae09b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:57.108880 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e4b4c75b-b91b-4b90-aba9-54f363b97205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:57.221237 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:57.330843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b4c75b-b91b-4b90-aba9-54f363b97205 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:57.334152 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c54a09-e1f3-48b2-b87e-33305bcae09b service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:57.341142 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0416b9d-74f7-4696-8a8f-456dd7ff4e61 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf5e2b4f5e440a7942adea3d51cec75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:57.342792 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0416b9d-74f7-4696-8a8f-456dd7ff4e61 service placement] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:57.343267 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0416b9d-74f7-4696-8a8f-456dd7ff4e61 service placement] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:57.343891 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0416b9d-74f7-4696-8a8f-456dd7ff4e61 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:57.346060 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f0416b9d-74f7-4696-8a8f-456dd7ff4e61 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:57.351239 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd1e8c4-a3b5-44c2-9e25-8121ed2100d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:57.351888 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1e8c4-a3b5-44c2-9e25-8121ed2100d6 service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:57.352215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1e8c4-a3b5-44c2-9e25-8121ed2100d6 service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:57.352536 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1e8c4-a3b5-44c2-9e25-8121ed2100d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:57.361562 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3dd1e8c4-a3b5-44c2-9e25-8121ed2100d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:57.364680 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0416b9d-74f7-4696-8a8f-456dd7ff4e61 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:57.367286 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0416b9d-74f7-4696-8a8f-456dd7ff4e61 service placement] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:57.391328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 871/1765] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:51:57.396266 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0416b9d-74f7-4696-8a8f-456dd7ff4e61 service placement] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:57.406787 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f0416b9d-74f7-4696-8a8f-456dd7ff4e61 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:57.624512 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:57.665042 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0416b9d-74f7-4696-8a8f-456dd7ff4e61 service placement] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:57.677993 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07ae16-ac28-4499-adb6-2634ce94e722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:57.682414 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07ae16-ac28-4499-adb6-2634ce94e722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:57.683198 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07ae16-ac28-4499-adb6-2634ce94e722 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:57.684395 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07ae16-ac28-4499-adb6-2634ce94e722 admin admin] PATH_INFO: `/v3/projects/ae6f3435f3e24ff3a5092d863daadbe8` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:57.689907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f07ae16-ac28-4499-adb6-2634ce94e722 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae6f3435f3e24ff3a5092d863daadbe8)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:57.695774 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f07ae16-ac28-4499-adb6-2634ce94e722 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:57.754390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd1e8c4-a3b5-44c2-9e25-8121ed2100d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:57.758691 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd1e8c4-a3b5-44c2-9e25-8121ed2100d6 service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:57.785351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 895/1766] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:57 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 681 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:51:57.786813 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd1e8c4-a3b5-44c2-9e25-8121ed2100d6 service nova] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:57.789164 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3dd1e8c4-a3b5-44c2-9e25-8121ed2100d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:57.847146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd1e8c4-a3b5-44c2-9e25-8121ed2100d6 service nova] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:57.857893 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3f07ae16-ac28-4499-adb6-2634ce94e722 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae6f3435f3e24ff3a5092d863daadbe8', 'request_id': 'req-3f07ae16-ac28-4499-adb6-2634ce94e722'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:57.876725 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3f07ae16-ac28-4499-adb6-2634ce94e722 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae6f3435f3e24ff3a5092d863daadbe8', 'request_id': 'req-3f07ae16-ac28-4499-adb6-2634ce94e722'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:57.893677 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3f07ae16-ac28-4499-adb6-2634ce94e722 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae6f3435f3e24ff3a5092d863daadbe8', 'request_id': 'req-3f07ae16-ac28-4499-adb6-2634ce94e722'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:57.935823 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3f07ae16-ac28-4499-adb6-2634ce94e722 admin admin] The token cache is being invalidate because project ae6f3435f3e24ff3a5092d863daadbe8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:51:57.937222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3f07ae16-ac28-4499-adb6-2634ce94e722 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:57.941455 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3f07ae16-ac28-4499-adb6-2634ce94e722 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:57.944720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 872/1767] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:51:57 2022] DELETE /identity/v3/projects/ae6f3435f3e24ff3a5092d863daadbe8 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:57.952450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07ae16-ac28-4499-adb6-2634ce94e722 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:57.955163 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3f07ae16-ac28-4499-adb6-2634ce94e722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:58.045504 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1083a742-1d3e-4bd7-bce0-781847a630d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:58.049213 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1083a742-1d3e-4bd7-bce0-781847a630d4 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:58.049356 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1083a742-1d3e-4bd7-bce0-781847a630d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:58.049775 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1083a742-1d3e-4bd7-bce0-781847a630d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:58.062368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1083a742-1d3e-4bd7-bce0-781847a630d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:58.125273 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1083a742-1d3e-4bd7-bce0-781847a630d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:58.135892 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1083a742-1d3e-4bd7-bce0-781847a630d4 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:58.188622 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 896/1768] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:51:58.193246 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d937679-79c3-4515-96a7-be365cf7f365 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:58.193538 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d937679-79c3-4515-96a7-be365cf7f365 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:58.193816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d937679-79c3-4515-96a7-be365cf7f365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:58.298926 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-6d937679-79c3-4515-96a7-be365cf7f365 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:58.306398 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-6d937679-79c3-4515-96a7-be365cf7f365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:58.322841 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07ae16-ac28-4499-adb6-2634ce94e722 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:58.351301 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc6981-0082-4834-8b34-a8bc53c9f137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:58.354715 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc6981-0082-4834-8b34-a8bc53c9f137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:58.354949 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc6981-0082-4834-8b34-a8bc53c9f137 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:58.356031 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc6981-0082-4834-8b34-a8bc53c9f137 admin admin] PATH_INFO: `/v3/users/eb62cd1f714c4b89937916b2179163db` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:58.356177 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 897/1769] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:58.356942 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc6981-0082-4834-8b34-a8bc53c9f137 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb62cd1f714c4b89937916b2179163db)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:58.359419 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc6981-0082-4834-8b34-a8bc53c9f137 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:58.371117 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268cb325-d875-404c-90e6-6be0f4b16b33 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:58.371510 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268cb325-d875-404c-90e6-6be0f4b16b33 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:58.371753 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268cb325-d875-404c-90e6-6be0f4b16b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:58.382862 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:58.415603 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-268cb325-d875-404c-90e6-6be0f4b16b33 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:58.417976 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-268cb325-d875-404c-90e6-6be0f4b16b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:58.435598 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 898/1770] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:58.441778 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268cb325-d875-404c-90e6-6be0f4b16b33 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:58.461520 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268cb325-d875-404c-90e6-6be0f4b16b33 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:58.483403 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0fdc0-a5e3-42de-98be-a7236e51169b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:58.485329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0fdc0-a5e3-42de-98be-a7236e51169b admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:58.497744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0fdc0-a5e3-42de-98be-a7236e51169b admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:58.500524 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0fdc0-a5e3-42de-98be-a7236e51169b admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:58.504031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c0fdc0-a5e3-42de-98be-a7236e51169b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:51:58.504826 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c0fdc0-a5e3-42de-98be-a7236e51169b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:58.526913 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c0fdc0-a5e3-42de-98be-a7236e51169b admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:58.529096 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-f7cc6981-0082-4834-8b34-a8bc53c9f137 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb62cd1f714c4b89937916b2179163db'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:58.553703 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-f7cc6981-0082-4834-8b34-a8bc53c9f137 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb62cd1f714c4b89937916b2179163db'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:58.613416 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-f7cc6981-0082-4834-8b34-a8bc53c9f137 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb62cd1f714c4b89937916b2179163db'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:58.618360 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-f7cc6981-0082-4834-8b34-a8bc53c9f137 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb62cd1f714c4b89937916b2179163db'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:58.620939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-f7cc6981-0082-4834-8b34-a8bc53c9f137 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb62cd1f714c4b89937916b2179163db'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:51:58.626326 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 899/1771] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:51:58.629512 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0fdc0-a5e3-42de-98be-a7236e51169b admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:58.639179 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-82c0fdc0-a5e3-42de-98be-a7236e51169b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:58.714640 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0fdc0-a5e3-42de-98be-a7236e51169b admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:58.748240 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee18ec5-4104-42d2-a213-8292bed34152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:58.749386 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee18ec5-4104-42d2-a213-8292bed34152 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:58.749841 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee18ec5-4104-42d2-a213-8292bed34152 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:58.750661 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee18ec5-4104-42d2-a213-8292bed34152 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:58.751592 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee18ec5-4104-42d2-a213-8292bed34152 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:58.755516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee18ec5-4104-42d2-a213-8292bed34152 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:58.790145 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 873/1772] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:51:57 2022] DELETE /identity/v3/users/eb62cd1f714c4b89937916b2179163db => generated 0 bytes in 839 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:51:58.793187 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee526c6-e185-4483-8c49-f3f670e71cf8 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:58.798989 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee526c6-e185-4483-8c49-f3f670e71cf8 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:58.807244 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee526c6-e185-4483-8c49-f3f670e71cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:58.907743 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 900/1773] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:51:58 2022] POST /identity/v3/projects => generated 393 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:58.916589 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee18ec5-4104-42d2-a213-8292bed34152 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:58.951575 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-dee526c6-e185-4483-8c49-f3f670e71cf8 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:58.980958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-dee526c6-e185-4483-8c49-f3f670e71cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:59.310674 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee18ec5-4104-42d2-a213-8292bed34152 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:59.323504 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a77988-cdaf-4645-9144-5d283f858005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:59.325648 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a77988-cdaf-4645-9144-5d283f858005 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:59.326068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a77988-cdaf-4645-9144-5d283f858005 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:59.326482 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a77988-cdaf-4645-9144-5d283f858005 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:59.328613 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a77988-cdaf-4645-9144-5d283f858005 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:59.335996 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a77988-cdaf-4645-9144-5d283f858005 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:59.390760 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 874/1774] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 595 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:51:59.393547 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31266371-df04-4ed5-844e-faf6c7897226 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:59.394014 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31266371-df04-4ed5-844e-faf6c7897226 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:59.401839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 901/1775] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:51:58 2022] POST /identity/v3/users => generated 297 bytes in 484 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:51:59.402034 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31266371-df04-4ed5-844e-faf6c7897226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:59.404284 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a77988-cdaf-4645-9144-5d283f858005 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:59.439380 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a77988-cdaf-4645-9144-5d283f858005 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:59.467356 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349d7ca-a912-43cc-9916-053db7ac050e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:51:59.468568 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349d7ca-a912-43cc-9916-053db7ac050e admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:51:59.471872 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349d7ca-a912-43cc-9916-053db7ac050e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:51:59.472415 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3349d7ca-a912-43cc-9916-053db7ac050e admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:51:59.474208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349d7ca-a912-43cc-9916-053db7ac050e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:51:59.477048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3349d7ca-a912-43cc-9916-053db7ac050e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:51:59.517358 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 902/1776] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:51:59.521665 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349d7ca-a912-43cc-9916-053db7ac050e admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:51:59.527662 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3349d7ca-a912-43cc-9916-053db7ac050e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:59.595984 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:51:59.650678 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-31266371-df04-4ed5-844e-faf6c7897226 None None] MFA Rules not processed for user `3aced0ed17b946f9961d87a1d4446c37`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:51:59.661156 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-31266371-df04-4ed5-844e-faf6c7897226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:51:59.956516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3349d7ca-a912-43cc-9916-053db7ac050e admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:51:59.999277 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5803b60-edee-4504-a966-5ab4becf4be5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:00.008694 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5803b60-edee-4504-a966-5ab4becf4be5 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:00.010087 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5803b60-edee-4504-a966-5ab4becf4be5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:00.011013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5803b60-edee-4504-a966-5ab4becf4be5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:00.057435 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5803b60-edee-4504-a966-5ab4becf4be5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:00.065103 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5803b60-edee-4504-a966-5ab4becf4be5 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:00.118940 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 875/1777] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:51:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 719 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:00.118940 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31266371-df04-4ed5-844e-faf6c7897226 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:00.137029 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 903/1778] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:51:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:00.144848 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad308cb-fdb9-41a0-964b-bb9478814b27 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:00.145147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad308cb-fdb9-41a0-964b-bb9478814b27 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:00.145431 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad308cb-fdb9-41a0-964b-bb9478814b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:00.182022 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31266371-df04-4ed5-844e-faf6c7897226 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:00.202399 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367048c-53d3-4f47-80ef-c09028eda565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:00.203714 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367048c-53d3-4f47-80ef-c09028eda565 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:00.210551 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367048c-53d3-4f47-80ef-c09028eda565 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:00.211013 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367048c-53d3-4f47-80ef-c09028eda565 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:00.230483 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367048c-53d3-4f47-80ef-c09028eda565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:00.232912 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367048c-53d3-4f47-80ef-c09028eda565 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:00.234313 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-cad308cb-fdb9-41a0-964b-bb9478814b27 None None] MFA Rules not processed for user `3aced0ed17b946f9961d87a1d4446c37`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:00.250937 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-cad308cb-fdb9-41a0-964b-bb9478814b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:00.292192 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 876/1779] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:00.304452 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 904/1780] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:00.311819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367048c-53d3-4f47-80ef-c09028eda565 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:00.329858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad308cb-fdb9-41a0-964b-bb9478814b27 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:00.330534 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:00.332523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-cad308cb-fdb9-41a0-964b-bb9478814b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:00.366131 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367048c-53d3-4f47-80ef-c09028eda565 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:00.393693 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c213426-0046-4181-94fc-2dc1ff1a9620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:00.394728 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c213426-0046-4181-94fc-2dc1ff1a9620 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:00.397202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c213426-0046-4181-94fc-2dc1ff1a9620 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:00.397961 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c213426-0046-4181-94fc-2dc1ff1a9620 admin admin] PATH_INFO: `/v3/projects/a31c296151c34140ab45ec45754ef9c3/users/544ba9e2043a41898427f576e5833fac/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:00.457208 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c213426-0046-4181-94fc-2dc1ff1a9620 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a31c296151c34140ab45ec45754ef9c3, user_id=544ba9e2043a41898427f576e5833fac, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:00.466447 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c213426-0046-4181-94fc-2dc1ff1a9620 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:00.548602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 877/1781] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:00 2022] PUT /identity/v3/projects/a31c296151c34140ab45ec45754ef9c3/users/544ba9e2043a41898427f576e5833fac/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:00.569976 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658e4a9-d111-4991-9a32-6053aa9bf1e7 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:00.570332 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658e4a9-d111-4991-9a32-6053aa9bf1e7 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:00.571058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658e4a9-d111-4991-9a32-6053aa9bf1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:00.661847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad308cb-fdb9-41a0-964b-bb9478814b27 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:00.693111 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b658e4a9-d111-4991-9a32-6053aa9bf1e7 None None] MFA Rules not processed for user `544ba9e2043a41898427f576e5833fac`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:00.697412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b658e4a9-d111-4991-9a32-6053aa9bf1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:01.014276 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 878/1782] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:01.022270 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b658e4a9-d111-4991-9a32-6053aa9bf1e7 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:01.104472 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01b51f-39b8-4f2f-a978-84098b062902 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:01.111383 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01b51f-39b8-4f2f-a978-84098b062902 service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:01.116831 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01b51f-39b8-4f2f-a978-84098b062902 service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:01.119965 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01b51f-39b8-4f2f-a978-84098b062902 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:01.279382 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:01.330043 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed01b51f-39b8-4f2f-a978-84098b062902 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:01.332075 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed01b51f-39b8-4f2f-a978-84098b062902 service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:01.349090 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b658e4a9-d111-4991-9a32-6053aa9bf1e7 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:01.371688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be64a5-b3c1-4826-98e3-858eecf7c9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:01.372446 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be64a5-b3c1-4826-98e3-858eecf7c9fc admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:01.372558 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 905/1783] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:00 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 1045 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:01.372751 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be64a5-b3c1-4826-98e3-858eecf7c9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:01.374208 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be64a5-b3c1-4826-98e3-858eecf7c9fc admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:01.374208 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6be64a5-b3c1-4826-98e3-858eecf7c9fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:01.383393 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6be64a5-b3c1-4826-98e3-858eecf7c9fc admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:01.516337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 879/1784] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:52:01 2022] POST /identity/v3/projects => generated 391 bytes in 495 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:01.521821 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01b51f-39b8-4f2f-a978-84098b062902 service nova] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:01.566301 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01b51f-39b8-4f2f-a978-84098b062902 service nova] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:01.823493 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf5bfb-f27d-4fd1-b1e2-f34c389bc1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:01.824355 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf5bfb-f27d-4fd1-b1e2-f34c389bc1e5 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:01.824893 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf5bfb-f27d-4fd1-b1e2-f34c389bc1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:01.825758 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf5bfb-f27d-4fd1-b1e2-f34c389bc1e5 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:01.828724 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf5bfb-f27d-4fd1-b1e2-f34c389bc1e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:01.832984 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf5bfb-f27d-4fd1-b1e2-f34c389bc1e5 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:01.917055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 906/1785] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:52:01 2022] POST /identity/v3/users => generated 302 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:01.927044 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be64a5-b3c1-4826-98e3-858eecf7c9fc admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:01.963294 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be64a5-b3c1-4826-98e3-858eecf7c9fc admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:01.990002 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf5bfb-f27d-4fd1-b1e2-f34c389bc1e5 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:01.992276 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ef960-7035-410d-9e39-5de77b7aa939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:01.992894 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-9adf5bfb-f27d-4fd1-b1e2-f34c389bc1e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:01.993291 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ef960-7035-410d-9e39-5de77b7aa939 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:01.995339 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ef960-7035-410d-9e39-5de77b7aa939 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:01.995918 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ef960-7035-410d-9e39-5de77b7aa939 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:01.997377 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ef960-7035-410d-9e39-5de77b7aa939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:02.007747 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ef960-7035-410d-9e39-5de77b7aa939 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:02.057271 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf5bfb-f27d-4fd1-b1e2-f34c389bc1e5 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:02.075696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 880/1786] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:02.077793 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ef960-7035-410d-9e39-5de77b7aa939 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:02.091759 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28731a21-b8c9-44f3-9320-26949ff07357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:02.100900 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28731a21-b8c9-44f3-9320-26949ff07357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:02.101223 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28731a21-b8c9-44f3-9320-26949ff07357 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:02.101600 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28731a21-b8c9-44f3-9320-26949ff07357 admin admin] PATH_INFO: `/v3/projects/e848b3909e6b43bfa5b7c8b8e344f1e1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:02.113516 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ef960-7035-410d-9e39-5de77b7aa939 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:02.115739 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28731a21-b8c9-44f3-9320-26949ff07357 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e848b3909e6b43bfa5b7c8b8e344f1e1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:02.130009 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28731a21-b8c9-44f3-9320-26949ff07357 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:02.140325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d562acb-fb38-4009-b864-8a58ff2cf05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:02.142736 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d562acb-fb38-4009-b864-8a58ff2cf05e admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:02.144251 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d562acb-fb38-4009-b864-8a58ff2cf05e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:02.145029 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d562acb-fb38-4009-b864-8a58ff2cf05e admin admin] PATH_INFO: `/v3/projects/82de73713cad4cd9ace2db1a039d5398/users/ba08c216301746d2be55a2759d763897/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:02.200326 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d562acb-fb38-4009-b864-8a58ff2cf05e admin admin] RBAC: Authorizing `identity:create_grant(project_id=82de73713cad4cd9ace2db1a039d5398, user_id=ba08c216301746d2be55a2759d763897, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:02.205516 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d562acb-fb38-4009-b864-8a58ff2cf05e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:02.241731 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 881/1787] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:02 2022] PUT /identity/v3/projects/82de73713cad4cd9ace2db1a039d5398/users/ba08c216301746d2be55a2759d763897/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:02.248379 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d562acb-fb38-4009-b864-8a58ff2cf05e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:02.382365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:02.410142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-28731a21-b8c9-44f3-9320-26949ff07357 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e848b3909e6b43bfa5b7c8b8e344f1e1', 'request_id': 'req-28731a21-b8c9-44f3-9320-26949ff07357'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:02.432706 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-28731a21-b8c9-44f3-9320-26949ff07357 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e848b3909e6b43bfa5b7c8b8e344f1e1', 'request_id': 'req-28731a21-b8c9-44f3-9320-26949ff07357'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:02.447584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-28731a21-b8c9-44f3-9320-26949ff07357 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e848b3909e6b43bfa5b7c8b8e344f1e1', 'request_id': 'req-28731a21-b8c9-44f3-9320-26949ff07357'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:02.479964 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-28731a21-b8c9-44f3-9320-26949ff07357 admin admin] The token cache is being invalidate because project e848b3909e6b43bfa5b7c8b8e344f1e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:52:02.480253 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-28731a21-b8c9-44f3-9320-26949ff07357 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:02.484181 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-28731a21-b8c9-44f3-9320-26949ff07357 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:02.494654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 907/1788] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:52:01 2022] DELETE /identity/v3/projects/e848b3909e6b43bfa5b7c8b8e344f1e1 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:52:02.496793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28731a21-b8c9-44f3-9320-26949ff07357 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:02.505534 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-28731a21-b8c9-44f3-9320-26949ff07357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:02.539829 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d562acb-fb38-4009-b864-8a58ff2cf05e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:02.959019 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:02.972294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28731a21-b8c9-44f3-9320-26949ff07357 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:03.008035 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f6820-8d93-4cb5-a15a-28dcb79494ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:03.014646 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f6820-8d93-4cb5-a15a-28dcb79494ea admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:03.018203 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8157e05-8669-416e-b2f3-bf9d24ab8435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:03.019063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8157e05-8669-416e-b2f3-bf9d24ab8435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:03.019386 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8157e05-8669-416e-b2f3-bf9d24ab8435 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:03.019673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8157e05-8669-416e-b2f3-bf9d24ab8435 admin admin] PATH_INFO: `/v3/users/af2bb237205c4b3e80ed68541ff40972` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:03.020768 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f6820-8d93-4cb5-a15a-28dcb79494ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:03.021189 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8157e05-8669-416e-b2f3-bf9d24ab8435 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af2bb237205c4b3e80ed68541ff40972)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:03.021340 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f6820-8d93-4cb5-a15a-28dcb79494ea admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:03.022452 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f6820-8d93-4cb5-a15a-28dcb79494ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:03.029651 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f6820-8d93-4cb5-a15a-28dcb79494ea admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:03.037269 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8157e05-8669-416e-b2f3-bf9d24ab8435 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:03.096575 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 882/1789] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 849 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:03.116757 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f6820-8d93-4cb5-a15a-28dcb79494ea admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:03.126047 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-175f6820-8d93-4cb5-a15a-28dcb79494ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:03.225587 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f6820-8d93-4cb5-a15a-28dcb79494ea admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:03.268915 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee329261-45ab-45ae-a753-e86f2db875aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:03.270102 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee329261-45ab-45ae-a753-e86f2db875aa admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:03.270582 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee329261-45ab-45ae-a753-e86f2db875aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:03.270990 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee329261-45ab-45ae-a753-e86f2db875aa admin admin] PATH_INFO: `/v3/projects/82de73713cad4cd9ace2db1a039d5398/users/ba08c216301746d2be55a2759d763897/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:03.272928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d8157e05-8669-416e-b2f3-bf9d24ab8435 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af2bb237205c4b3e80ed68541ff40972'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:03.293162 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee329261-45ab-45ae-a753-e86f2db875aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=82de73713cad4cd9ace2db1a039d5398, user_id=ba08c216301746d2be55a2759d763897, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:03.312557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee329261-45ab-45ae-a753-e86f2db875aa admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:03.361144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d8157e05-8669-416e-b2f3-bf9d24ab8435 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af2bb237205c4b3e80ed68541ff40972'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:03.410620 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 883/1790] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:03 2022] PUT /identity/v3/projects/82de73713cad4cd9ace2db1a039d5398/users/ba08c216301746d2be55a2759d763897/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:03.413076 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d8157e05-8669-416e-b2f3-bf9d24ab8435 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af2bb237205c4b3e80ed68541ff40972'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:03.430346 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acfca0-8265-469d-ac3f-f1f52dcfe478 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:03.431322 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acfca0-8265-469d-ac3f-f1f52dcfe478 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:03.435191 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acfca0-8265-469d-ac3f-f1f52dcfe478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:03.455721 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d8157e05-8669-416e-b2f3-bf9d24ab8435 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af2bb237205c4b3e80ed68541ff40972'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:03.461803 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d8157e05-8669-416e-b2f3-bf9d24ab8435 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af2bb237205c4b3e80ed68541ff40972'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:03.513128 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 908/1791] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:52:02 2022] DELETE /identity/v3/users/af2bb237205c4b3e80ed68541ff40972 => generated 0 bytes in 1017 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:03.592045 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-e7acfca0-8265-469d-ac3f-f1f52dcfe478 None None] MFA Rules not processed for user `ba08c216301746d2be55a2759d763897`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:03.606762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e7acfca0-8265-469d-ac3f-f1f52dcfe478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:04.036117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 884/1792] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:03 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 608 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:04.051395 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8157e05-8669-416e-b2f3-bf9d24ab8435 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:04.053285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d8157e05-8669-416e-b2f3-bf9d24ab8435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:04.185917 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8157e05-8669-416e-b2f3-bf9d24ab8435 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:04.193426 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047590ea-d284-446f-a73d-763aceed7b3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:04.195236 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047590ea-d284-446f-a73d-763aceed7b3f service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:04.197255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047590ea-d284-446f-a73d-763aceed7b3f service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:04.198042 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047590ea-d284-446f-a73d-763aceed7b3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:04.240276 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047590ea-d284-446f-a73d-763aceed7b3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:04.245508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047590ea-d284-446f-a73d-763aceed7b3f service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:04.272383 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 909/1793] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:04 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:04.488231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acfca0-8265-469d-ac3f-f1f52dcfe478 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:04.510688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acfca0-8265-469d-ac3f-f1f52dcfe478 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:04.534382 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afdacb1-a54a-40c8-9bec-3563f100972a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:04.535075 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afdacb1-a54a-40c8-9bec-3563f100972a service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:04.535346 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afdacb1-a54a-40c8-9bec-3563f100972a service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:04.535623 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afdacb1-a54a-40c8-9bec-3563f100972a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:04.546083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5afdacb1-a54a-40c8-9bec-3563f100972a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:04.689485 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:04.766506 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afdacb1-a54a-40c8-9bec-3563f100972a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:04.769507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afdacb1-a54a-40c8-9bec-3563f100972a service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:04.801295 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 885/1794] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:04 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:04.804660 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ccaf5-9aed-49d4-b82f-085398bc1850 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:04.805441 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ccaf5-9aed-49d4-b82f-085398bc1850 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:04.805441 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ccaf5-9aed-49d4-b82f-085398bc1850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:04.980935 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e9e91-cd92-4636-a520-51e11e13e424 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:04.982724 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e9e91-cd92-4636-a520-51e11e13e424 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:04.983277 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e9e91-cd92-4636-a520-51e11e13e424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:05.018784 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-1b6ccaf5-9aed-49d4-b82f-085398bc1850 None None] MFA Rules not processed for user `df07cffd82c74621855c8b8d7bfa989d`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:05.045923 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1b6ccaf5-9aed-49d4-b82f-085398bc1850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:05.088532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-d83e9e91-cd92-4636-a520-51e11e13e424 None None] MFA Rules not processed for user `63e1c5a24be04ca4abd5d5109c5763f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:05.090432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d83e9e91-cd92-4636-a520-51e11e13e424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:05.255308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:05.315058 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 910/1795] 10.222.0.71 () {58 vars in 1009 bytes} [Fri Aug 26 08:52:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 511 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:05.319391 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56d5a0-6dcd-4aa7-aae4-efe3bd5327c3 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:05.327061 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56d5a0-6dcd-4aa7-aae4-efe3bd5327c3 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:05.327628 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56d5a0-6dcd-4aa7-aae4-efe3bd5327c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:05.365687 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-bc56d5a0-6dcd-4aa7-aae4-efe3bd5327c3 None None] MFA Rules not processed for user `3aced0ed17b946f9961d87a1d4446c37`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:05.367652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-bc56d5a0-6dcd-4aa7-aae4-efe3bd5327c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:05.458002 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 911/1796] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:05 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:05.475947 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 886/1797] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 496 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:52:05.584997 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc56d5a0-6dcd-4aa7-aae4-efe3bd5327c3 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:05.588988 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-bc56d5a0-6dcd-4aa7-aae4-efe3bd5327c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:05.874767 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc56d5a0-6dcd-4aa7-aae4-efe3bd5327c3 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:05.893946 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d049d-c478-4932-b0de-3f2bc914dc45 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1723e961375b45b7b3bd99ddeb69683f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:05.894838 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d049d-c478-4932-b0de-3f2bc914dc45 service swift] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:05.895223 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d049d-c478-4932-b0de-3f2bc914dc45 service swift] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:05.895768 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725d049d-c478-4932-b0de-3f2bc914dc45 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:05.936228 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d049d-c478-4932-b0de-3f2bc914dc45 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:05.942377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725d049d-c478-4932-b0de-3f2bc914dc45 service swift] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:05.989636 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 912/1798] 10.222.0.71 () {60 vars in 1384 bytes} [Fri Aug 26 08:52:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 08:52:06.366137 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e9e91-cd92-4636-a520-51e11e13e424 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:06.373992 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d83e9e91-cd92-4636-a520-51e11e13e424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:06.425944 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d049d-c478-4932-b0de-3f2bc914dc45 service swift] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:06.428796 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-725d049d-c478-4932-b0de-3f2bc914dc45 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:06.698095 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e9e91-cd92-4636-a520-51e11e13e424 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:06.715733 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93a439-6ca2-441c-b8fd-29c11478dcb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:06.719474 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93a439-6ca2-441c-b8fd-29c11478dcb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:06.721020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725d049d-c478-4932-b0de-3f2bc914dc45 service swift] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:06.721545 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93a439-6ca2-441c-b8fd-29c11478dcb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:06.722141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93a439-6ca2-441c-b8fd-29c11478dcb3 admin admin] PATH_INFO: `/v3/projects/e2593c59fb474db69cfce5a98d514c75` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:06.725588 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f93a439-6ca2-441c-b8fd-29c11478dcb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2593c59fb474db69cfce5a98d514c75)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:06.730253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f93a439-6ca2-441c-b8fd-29c11478dcb3 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:06.743815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe5f4a-35af-4fc9-bbdf-ba5ec661924d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:06.744989 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe5f4a-35af-4fc9-bbdf-ba5ec661924d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:06.745464 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe5f4a-35af-4fc9-bbdf-ba5ec661924d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:06.749164 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe5f4a-35af-4fc9-bbdf-ba5ec661924d admin admin] PATH_INFO: `/v3/users/7b18e70d8d114e62996b0ed0b96f55ea` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:06.759954 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe5f4a-35af-4fc9-bbdf-ba5ec661924d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b18e70d8d114e62996b0ed0b96f55ea)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:06.764426 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe5f4a-35af-4fc9-bbdf-ba5ec661924d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:06.789591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:06.933201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5f93a439-6ca2-441c-b8fd-29c11478dcb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2593c59fb474db69cfce5a98d514c75', 'request_id': 'req-5f93a439-6ca2-441c-b8fd-29c11478dcb3'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:06.942226 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-1abe5f4a-35af-4fc9-bbdf-ba5ec661924d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b18e70d8d114e62996b0ed0b96f55ea'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:06.961032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5f93a439-6ca2-441c-b8fd-29c11478dcb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2593c59fb474db69cfce5a98d514c75', 'request_id': 'req-5f93a439-6ca2-441c-b8fd-29c11478dcb3'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:06.989709 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5f93a439-6ca2-441c-b8fd-29c11478dcb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2593c59fb474db69cfce5a98d514c75', 'request_id': 'req-5f93a439-6ca2-441c-b8fd-29c11478dcb3'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:07.009911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-1abe5f4a-35af-4fc9-bbdf-ba5ec661924d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b18e70d8d114e62996b0ed0b96f55ea'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:07.043541 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5f93a439-6ca2-441c-b8fd-29c11478dcb3 admin admin] The token cache is being invalidate because project e2593c59fb474db69cfce5a98d514c75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:52:07.043846 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5f93a439-6ca2-441c-b8fd-29c11478dcb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:07.045400 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5f93a439-6ca2-441c-b8fd-29c11478dcb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:07.046222 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-1abe5f4a-35af-4fc9-bbdf-ba5ec661924d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b18e70d8d114e62996b0ed0b96f55ea'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:07.050968 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 887/1799] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:52:06 2022] DELETE /identity/v3/projects/e2593c59fb474db69cfce5a98d514c75 => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:07.058114 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ca417-7d7a-4a5d-97f3-0d8984829e3c None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:07.058516 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ca417-7d7a-4a5d-97f3-0d8984829e3c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:07.058798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ca417-7d7a-4a5d-97f3-0d8984829e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:07.068134 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-1abe5f4a-35af-4fc9-bbdf-ba5ec661924d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b18e70d8d114e62996b0ed0b96f55ea'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:07.070283 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-1abe5f4a-35af-4fc9-bbdf-ba5ec661924d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b18e70d8d114e62996b0ed0b96f55ea'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:07.099192 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 913/1800] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:52:06 2022] DELETE /identity/v3/users/7b18e70d8d114e62996b0ed0b96f55ea => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:07.111759 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:07.121701 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c41fa-d0db-4720-9276-088e41809a22 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:07.122349 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c41fa-d0db-4720-9276-088e41809a22 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:07.122825 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c41fa-d0db-4720-9276-088e41809a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:07.144531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ec9ca417-7d7a-4a5d-97f3-0d8984829e3c None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:07.147541 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ec9ca417-7d7a-4a5d-97f3-0d8984829e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:07.160429 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-d39c41fa-d0db-4720-9276-088e41809a22 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:07.162907 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d39c41fa-d0db-4720-9276-088e41809a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:07.427717 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 888/1801] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:07.457134 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 914/1802] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:07.461836 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4453b-3bcb-4dbc-a29c-e8115a07c2a6 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:07.463631 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4453b-3bcb-4dbc-a29c-e8115a07c2a6 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:07.465099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4453b-3bcb-4dbc-a29c-e8115a07c2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:07.508914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-1ce4453b-3bcb-4dbc-a29c-e8115a07c2a6 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:07.510735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1ce4453b-3bcb-4dbc-a29c-e8115a07c2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:07.537031 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 889/1803] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:07.541622 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c41fa-d0db-4720-9276-088e41809a22 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:07.575812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c41fa-d0db-4720-9276-088e41809a22 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:07.591335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce4453b-3bcb-4dbc-a29c-e8115a07c2a6 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:07.593400 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1ce4453b-3bcb-4dbc-a29c-e8115a07c2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:07.606187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c0d9f-50d4-49c4-97da-c4f9ce8e769b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:07.606835 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c0d9f-50d4-49c4-97da-c4f9ce8e769b admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:07.607148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c0d9f-50d4-49c4-97da-c4f9ce8e769b admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:07.607420 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c0d9f-50d4-49c4-97da-c4f9ce8e769b admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:07.619876 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c0d9f-50d4-49c4-97da-c4f9ce8e769b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:52:07.624130 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c0d9f-50d4-49c4-97da-c4f9ce8e769b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:07.635628 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c0d9f-50d4-49c4-97da-c4f9ce8e769b admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:07.723410 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 915/1804] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:52:07.728766 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c0d9f-50d4-49c4-97da-c4f9ce8e769b admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:07.757626 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c0d9f-50d4-49c4-97da-c4f9ce8e769b admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:07.776306 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7147da8-0e8d-459d-9197-81beaec08b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:07.777453 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7147da8-0e8d-459d-9197-81beaec08b1e admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:07.777998 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7147da8-0e8d-459d-9197-81beaec08b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:07.778489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7147da8-0e8d-459d-9197-81beaec08b1e admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:07.779428 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7147da8-0e8d-459d-9197-81beaec08b1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:07.784079 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7147da8-0e8d-459d-9197-81beaec08b1e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:07.850152 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce4453b-3bcb-4dbc-a29c-e8115a07c2a6 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:07.861183 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114954d-0a5c-407a-8388-c6e92198cab1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:07.861954 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114954d-0a5c-407a-8388-c6e92198cab1 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:07.862965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114954d-0a5c-407a-8388-c6e92198cab1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:07.863411 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114954d-0a5c-407a-8388-c6e92198cab1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:07.865817 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 916/1805] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:52:07 2022] POST /identity/v3/projects => generated 381 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:52:07.872973 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7147da8-0e8d-459d-9197-81beaec08b1e admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:07.886352 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f7147da8-0e8d-459d-9197-81beaec08b1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:07.892721 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114954d-0a5c-407a-8388-c6e92198cab1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:07.895895 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114954d-0a5c-407a-8388-c6e92198cab1 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:07.934761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 890/1806] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:07.942362 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114954d-0a5c-407a-8388-c6e92198cab1 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:07.965655 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114954d-0a5c-407a-8388-c6e92198cab1 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:08.186854 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:08.255653 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7147da8-0e8d-459d-9197-81beaec08b1e admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:08.261859 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e90286-7ab4-4527-9bc8-031ee6d890e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:08.262523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e90286-7ab4-4527-9bc8-031ee6d890e1 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:08.262815 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e90286-7ab4-4527-9bc8-031ee6d890e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:08.263081 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e90286-7ab4-4527-9bc8-031ee6d890e1 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:08.263730 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e90286-7ab4-4527-9bc8-031ee6d890e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:08.266375 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e90286-7ab4-4527-9bc8-031ee6d890e1 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:08.277444 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d5e1a-a055-4a01-9daa-5e975deb33f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:08.278733 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d5e1a-a055-4a01-9daa-5e975deb33f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:08.279066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d5e1a-a055-4a01-9daa-5e975deb33f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:08.279343 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d5e1a-a055-4a01-9daa-5e975deb33f0 admin admin] PATH_INFO: `/v3/users/a733093a4907444899d8507117675b0b` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:08.280787 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d5e1a-a055-4a01-9daa-5e975deb33f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a733093a4907444899d8507117675b0b)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:08.283117 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d5e1a-a055-4a01-9daa-5e975deb33f0 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:08.337711 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 891/1807] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:52:07 2022] POST /identity/v3/users => generated 291 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:08.342009 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e90286-7ab4-4527-9bc8-031ee6d890e1 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:08.417031 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e90286-7ab4-4527-9bc8-031ee6d890e1 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:08.438660 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2ff36-07d3-49a2-ad4a-8d2481d73e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:08.439640 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2ff36-07d3-49a2-ad4a-8d2481d73e90 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:08.440122 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2ff36-07d3-49a2-ad4a-8d2481d73e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:08.440607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2ff36-07d3-49a2-ad4a-8d2481d73e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:08.441467 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f2ff36-07d3-49a2-ad4a-8d2481d73e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:08.443410 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f2ff36-07d3-49a2-ad4a-8d2481d73e90 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:08.489052 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 892/1808] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 08:52:08.494089 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2ff36-07d3-49a2-ad4a-8d2481d73e90 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:08.512165 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-808d5e1a-a055-4a01-9daa-5e975deb33f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a733093a4907444899d8507117675b0b'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:08.530846 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2ff36-07d3-49a2-ad4a-8d2481d73e90 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:08.559491 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21efafe1-4824-4f59-9119-ffe13167fbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:08.561855 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21efafe1-4824-4f59-9119-ffe13167fbd8 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:08.562377 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21efafe1-4824-4f59-9119-ffe13167fbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:08.562816 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21efafe1-4824-4f59-9119-ffe13167fbd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:08.564095 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21efafe1-4824-4f59-9119-ffe13167fbd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:08.575648 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21efafe1-4824-4f59-9119-ffe13167fbd8 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:08.585099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-808d5e1a-a055-4a01-9daa-5e975deb33f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a733093a4907444899d8507117675b0b'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:08.611545 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-808d5e1a-a055-4a01-9daa-5e975deb33f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a733093a4907444899d8507117675b0b'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:08.631934 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-808d5e1a-a055-4a01-9daa-5e975deb33f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a733093a4907444899d8507117675b0b'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:08.640073 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-808d5e1a-a055-4a01-9daa-5e975deb33f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a733093a4907444899d8507117675b0b'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:08.648940 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 893/1809] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:08.650365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21efafe1-4824-4f59-9119-ffe13167fbd8 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:08.660729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-21efafe1-4824-4f59-9119-ffe13167fbd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:08.686434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 917/1810] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:52:07 2022] DELETE /identity/v3/users/a733093a4907444899d8507117675b0b => generated 0 bytes in 816 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:08.693230 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855f1d8-ad71-4f06-837f-caa121caea68 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:08.693686 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855f1d8-ad71-4f06-837f-caa121caea68 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:08.693978 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855f1d8-ad71-4f06-837f-caa121caea68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:08.717678 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21efafe1-4824-4f59-9119-ffe13167fbd8 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:08.756139 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:08.787764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-1855f1d8-ad71-4f06-837f-caa121caea68 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:08.796517 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1855f1d8-ad71-4f06-837f-caa121caea68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:08.953598 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1931d-605b-4bf4-8672-45c12def1319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:08.954697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1931d-605b-4bf4-8672-45c12def1319 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:08.954829 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1931d-605b-4bf4-8672-45c12def1319 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:08.955330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1931d-605b-4bf4-8672-45c12def1319 admin admin] PATH_INFO: `/v3/projects/a63163b33b2c4f649ace39b8e22eb8bf/users/5d079fd7a2c946f19f5d8dbd446a7882/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:09.018275 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1931d-605b-4bf4-8672-45c12def1319 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a63163b33b2c4f649ace39b8e22eb8bf, user_id=5d079fd7a2c946f19f5d8dbd446a7882, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:09.021457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1931d-605b-4bf4-8672-45c12def1319 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:09.073174 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 894/1811] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:08 2022] PUT /identity/v3/projects/a63163b33b2c4f649ace39b8e22eb8bf/users/5d079fd7a2c946f19f5d8dbd446a7882/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:09.089723 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d2118-a6f7-4e94-8820-6729f0d0bd18 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:09.090165 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d2118-a6f7-4e94-8820-6729f0d0bd18 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:09.090600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d2118-a6f7-4e94-8820-6729f0d0bd18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:09.191445 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 918/1812] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 504 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:09.211414 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-041d2118-a6f7-4e94-8820-6729f0d0bd18 None None] MFA Rules not processed for user `5d079fd7a2c946f19f5d8dbd446a7882`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:09.211414 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-041d2118-a6f7-4e94-8820-6729f0d0bd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:09.394409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855f1d8-ad71-4f06-837f-caa121caea68 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:09.395786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1855f1d8-ad71-4f06-837f-caa121caea68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:09.446548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 895/1813] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:09.454064 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d2118-a6f7-4e94-8820-6729f0d0bd18 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:09.823488 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855f1d8-ad71-4f06-837f-caa121caea68 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:09.972419 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d2118-a6f7-4e94-8820-6729f0d0bd18 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:09.983715 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:10.024922 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e41711-59ce-42df-b825-7af205a13cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:10.028497 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e41711-59ce-42df-b825-7af205a13cea admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:10.041940 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e41711-59ce-42df-b825-7af205a13cea admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:10.042137 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e41711-59ce-42df-b825-7af205a13cea admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:10.044954 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e41711-59ce-42df-b825-7af205a13cea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:10.054704 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e41711-59ce-42df-b825-7af205a13cea admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:10.154719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077de04e-8925-426e-903e-7c6ca2e2c10a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:10.154719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077de04e-8925-426e-903e-7c6ca2e2c10a service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:10.154719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077de04e-8925-426e-903e-7c6ca2e2c10a service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:10.154719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077de04e-8925-426e-903e-7c6ca2e2c10a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:10.171249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 896/1814] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:52:09 2022] POST /identity/v3/projects => generated 383 bytes in 721 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:52:10.174687 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e41711-59ce-42df-b825-7af205a13cea admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:10.209455 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077de04e-8925-426e-903e-7c6ca2e2c10a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:10.212829 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077de04e-8925-426e-903e-7c6ca2e2c10a service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:10.239856 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:10.270552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 919/1815] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 878 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:10.575547 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e41711-59ce-42df-b825-7af205a13cea admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:10.598321 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac2a4a-84e2-45eb-b4fa-c4371231f5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:10.599993 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac2a4a-84e2-45eb-b4fa-c4371231f5c8 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:10.600733 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac2a4a-84e2-45eb-b4fa-c4371231f5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:10.601412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac2a4a-84e2-45eb-b4fa-c4371231f5c8 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:10.602781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac2a4a-84e2-45eb-b4fa-c4371231f5c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:10.605449 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac2a4a-84e2-45eb-b4fa-c4371231f5c8 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:10.696112 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 897/1816] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:52:10 2022] POST /identity/v3/users => generated 298 bytes in 522 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:10.701129 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077de04e-8925-426e-903e-7c6ca2e2c10a service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:10.728068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077de04e-8925-426e-903e-7c6ca2e2c10a service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:10.779199 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f8ce5-bb33-489c-9dac-90f55948d39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:10.780335 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f8ce5-bb33-489c-9dac-90f55948d39f admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:10.780819 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f8ce5-bb33-489c-9dac-90f55948d39f admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:10.781324 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f8ce5-bb33-489c-9dac-90f55948d39f admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:10.782560 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f8ce5-bb33-489c-9dac-90f55948d39f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:10.785869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f8ce5-bb33-489c-9dac-90f55948d39f admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:10.897766 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 920/1817] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:10.900331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac2a4a-84e2-45eb-b4fa-c4371231f5c8 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:10.965943 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac2a4a-84e2-45eb-b4fa-c4371231f5c8 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:11.007442 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf6dc9-f28e-45c6-99c4-7f547488fddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:11.013414 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf6dc9-f28e-45c6-99c4-7f547488fddc admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:11.016710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf6dc9-f28e-45c6-99c4-7f547488fddc admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:11.017358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf6dc9-f28e-45c6-99c4-7f547488fddc admin admin] PATH_INFO: `/v3/projects/12fbe05300254a7e83a7d03d1b95ce45/users/118e1b1cf4b648beae8ba3c221c640d2/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:11.099860 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cf6dc9-f28e-45c6-99c4-7f547488fddc admin admin] RBAC: Authorizing `identity:create_grant(project_id=12fbe05300254a7e83a7d03d1b95ce45, user_id=118e1b1cf4b648beae8ba3c221c640d2, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:11.104410 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cf6dc9-f28e-45c6-99c4-7f547488fddc admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:11.198162 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 898/1818] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:10 2022] PUT /identity/v3/projects/12fbe05300254a7e83a7d03d1b95ce45/users/118e1b1cf4b648beae8ba3c221c640d2/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:11.201587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f8ce5-bb33-489c-9dac-90f55948d39f admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:11.251105 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f8ce5-bb33-489c-9dac-90f55948d39f admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:11.593541 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98d438-5b51-45a9-99b4-3763f4f1dae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:11.594208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98d438-5b51-45a9-99b4-3763f4f1dae5 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:11.594495 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98d438-5b51-45a9-99b4-3763f4f1dae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:11.594771 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98d438-5b51-45a9-99b4-3763f4f1dae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:11.595506 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98d438-5b51-45a9-99b4-3763f4f1dae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:11.597217 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98d438-5b51-45a9-99b4-3763f4f1dae5 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:11.678054 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf6dc9-f28e-45c6-99c4-7f547488fddc admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:11.679018 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 921/1819] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 476 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:11.707093 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf6dc9-f28e-45c6-99c4-7f547488fddc admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:11.738268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26f65b2-8de3-4405-a787-0234133ef382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:11.742711 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f65b2-8de3-4405-a787-0234133ef382 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:11.745258 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f65b2-8de3-4405-a787-0234133ef382 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:11.746708 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f65b2-8de3-4405-a787-0234133ef382 admin admin] PATH_INFO: `/v3/projects/12fbe05300254a7e83a7d03d1b95ce45/users/118e1b1cf4b648beae8ba3c221c640d2/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:11.753399 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26f65b2-8de3-4405-a787-0234133ef382 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12fbe05300254a7e83a7d03d1b95ce45, user_id=118e1b1cf4b648beae8ba3c221c640d2, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:11.758929 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26f65b2-8de3-4405-a787-0234133ef382 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:11.829236 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 899/1820] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:11 2022] PUT /identity/v3/projects/12fbe05300254a7e83a7d03d1b95ce45/users/118e1b1cf4b648beae8ba3c221c640d2/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:11.845049 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281efa02-4a9d-4879-b432-27c706885e6e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:11.849470 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281efa02-4a9d-4879-b432-27c706885e6e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:11.850505 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281efa02-4a9d-4879-b432-27c706885e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:11.988361 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-281efa02-4a9d-4879-b432-27c706885e6e None None] MFA Rules not processed for user `118e1b1cf4b648beae8ba3c221c640d2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:11.989962 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-281efa02-4a9d-4879-b432-27c706885e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:12.164259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26f65b2-8de3-4405-a787-0234133ef382 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:12.168846 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e26f65b2-8de3-4405-a787-0234133ef382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:12.355769 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 922/1821] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:11 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 508 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:12.355769 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8db8b0-01fa-4d1a-aa48-d8afa3b5cfde None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:12.355769 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8db8b0-01fa-4d1a-aa48-d8afa3b5cfde None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:12.355769 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8db8b0-01fa-4d1a-aa48-d8afa3b5cfde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:12.425928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-3e8db8b0-01fa-4d1a-aa48-d8afa3b5cfde None None] MFA Rules not processed for user `4aa8938a504c4c4f8815478e520482d8`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:12.434132 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3e8db8b0-01fa-4d1a-aa48-d8afa3b5cfde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:12.565364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26f65b2-8de3-4405-a787-0234133ef382 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:12.581300 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa89c9f6-0565-48df-b0b8-63cbf63e96de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:12.582140 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89c9f6-0565-48df-b0b8-63cbf63e96de admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:12.582531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89c9f6-0565-48df-b0b8-63cbf63e96de admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:12.589722 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa89c9f6-0565-48df-b0b8-63cbf63e96de admin admin] PATH_INFO: `/v3/users/544ba9e2043a41898427f576e5833fac` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:12.592688 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:12.593460 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa89c9f6-0565-48df-b0b8-63cbf63e96de admin admin] RBAC: Authorizing `identity:delete_user(user_id=544ba9e2043a41898427f576e5833fac)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:12.595727 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa89c9f6-0565-48df-b0b8-63cbf63e96de admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:12.651050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 923/1822] 10.222.0.115 () {58 vars in 1016 bytes} [Fri Aug 26 08:52:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:12.651837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8db8b0-01fa-4d1a-aa48-d8afa3b5cfde None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:12.669130 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3e8db8b0-01fa-4d1a-aa48-d8afa3b5cfde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:12.720262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fa89c9f6-0565-48df-b0b8-63cbf63e96de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '544ba9e2043a41898427f576e5833fac'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:12.747669 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fa89c9f6-0565-48df-b0b8-63cbf63e96de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '544ba9e2043a41898427f576e5833fac'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:12.778545 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fa89c9f6-0565-48df-b0b8-63cbf63e96de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '544ba9e2043a41898427f576e5833fac'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:12.779986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fa89c9f6-0565-48df-b0b8-63cbf63e96de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '544ba9e2043a41898427f576e5833fac'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:12.780835 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fa89c9f6-0565-48df-b0b8-63cbf63e96de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '544ba9e2043a41898427f576e5833fac'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:12.782803 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:12.820943 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 900/1823] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:52:12 2022] DELETE /identity/v3/users/544ba9e2043a41898427f576e5833fac => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:52:12.825282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1553c3-cd11-44c2-87cb-7197c30d17d4 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:12.825834 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1553c3-cd11-44c2-87cb-7197c30d17d4 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:12.826385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1553c3-cd11-44c2-87cb-7197c30d17d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:12.868997 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8db8b0-01fa-4d1a-aa48-d8afa3b5cfde None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:12.882312 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-bb1553c3-cd11-44c2-87cb-7197c30d17d4 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:12.887889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-bb1553c3-cd11-44c2-87cb-7197c30d17d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:13.108170 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfaaf8e-ebc7-40ba-863a-90ef7023fce3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:13.109804 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfaaf8e-ebc7-40ba-863a-90ef7023fce3 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:13.110334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfaaf8e-ebc7-40ba-863a-90ef7023fce3 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:13.110960 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfaaf8e-ebc7-40ba-863a-90ef7023fce3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:13.123314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-bdfaaf8e-ebc7-40ba-863a-90ef7023fce3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:13.129498 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 901/1824] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:13.129498 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1553c3-cd11-44c2-87cb-7197c30d17d4 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:13.142860 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-bb1553c3-cd11-44c2-87cb-7197c30d17d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:13.305503 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1553c3-cd11-44c2-87cb-7197c30d17d4 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:13.334616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff0468-68f7-4d90-ae65-bd41b1ea08cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:13.336757 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff0468-68f7-4d90-ae65-bd41b1ea08cf service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:13.340602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff0468-68f7-4d90-ae65-bd41b1ea08cf service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:13.342531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff0468-68f7-4d90-ae65-bd41b1ea08cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:13.382531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ddff0468-68f7-4d90-ae65-bd41b1ea08cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:13.397346 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfaaf8e-ebc7-40ba-863a-90ef7023fce3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:13.408783 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfaaf8e-ebc7-40ba-863a-90ef7023fce3 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:13.445385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff0468-68f7-4d90-ae65-bd41b1ea08cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:13.448301 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff0468-68f7-4d90-ae65-bd41b1ea08cf service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:13.497091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 924/1825] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 845 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:13.508684 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfaaf8e-ebc7-40ba-863a-90ef7023fce3 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:13.537428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 902/1826] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:13.558706 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfaaf8e-ebc7-40ba-863a-90ef7023fce3 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:13.577857 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e255bd-190e-4c3b-9536-e69bcc7ed802 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:13.578652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e255bd-190e-4c3b-9536-e69bcc7ed802 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:13.579112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e255bd-190e-4c3b-9536-e69bcc7ed802 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:13.579481 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e255bd-190e-4c3b-9536-e69bcc7ed802 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:13.612450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e255bd-190e-4c3b-9536-e69bcc7ed802 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:13.623146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e255bd-190e-4c3b-9536-e69bcc7ed802 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:13.664993 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff0468-68f7-4d90-ae65-bd41b1ea08cf service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:13.666462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ddff0468-68f7-4d90-ae65-bd41b1ea08cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:13.679786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 925/1827] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:13.682117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff0468-68f7-4d90-ae65-bd41b1ea08cf service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:13.727420 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b2607-b3e2-459b-b585-76c337d58fb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:13.728215 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b2607-b3e2-459b-b585-76c337d58fb4 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:13.728418 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b2607-b3e2-459b-b585-76c337d58fb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:13.728615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b2607-b3e2-459b-b585-76c337d58fb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:13.761317 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e255bd-190e-4c3b-9536-e69bcc7ed802 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:13.764934 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-26e255bd-190e-4c3b-9536-e69bcc7ed802 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:13.784796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b2607-b3e2-459b-b585-76c337d58fb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:13.786515 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b2607-b3e2-459b-b585-76c337d58fb4 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:13.793625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e255bd-190e-4c3b-9536-e69bcc7ed802 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:13.802341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9258a9f-6f17-490b-b6c3-ee94f484c1cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:13.803315 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9258a9f-6f17-490b-b6c3-ee94f484c1cb service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:13.803812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9258a9f-6f17-490b-b6c3-ee94f484c1cb service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:13.804265 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9258a9f-6f17-490b-b6c3-ee94f484c1cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:13.807516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c9258a9f-6f17-490b-b6c3-ee94f484c1cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:13.819285 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 903/1828] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:52:13.883435 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:13.915436 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9258a9f-6f17-490b-b6c3-ee94f484c1cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:13.917661 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9258a9f-6f17-490b-b6c3-ee94f484c1cb service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:13.944015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 926/1829] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:52:14.277929 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b2607-b3e2-459b-b585-76c337d58fb4 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:14.288044 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ae4b2607-b3e2-459b-b585-76c337d58fb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:14.381052 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b2607-b3e2-459b-b585-76c337d58fb4 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:14.424568 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad6a50-1f7c-4bc7-a60d-4b3762c67bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:14.425596 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad6a50-1f7c-4bc7-a60d-4b3762c67bd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:14.426597 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad6a50-1f7c-4bc7-a60d-4b3762c67bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:14.428658 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad6a50-1f7c-4bc7-a60d-4b3762c67bd6 admin admin] PATH_INFO: `/v3/projects/7bccc002c73847b3b688c5eee1cef8b0` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:14.440669 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad6a50-1f7c-4bc7-a60d-4b3762c67bd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bccc002c73847b3b688c5eee1cef8b0)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:14.445735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad6a50-1f7c-4bc7-a60d-4b3762c67bd6 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:14.538768 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9258a9f-6f17-490b-b6c3-ee94f484c1cb service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:14.540627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c9258a9f-6f17-490b-b6c3-ee94f484c1cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:14.610456 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9258a9f-6f17-490b-b6c3-ee94f484c1cb service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:14.639674 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b7633-c94d-4cbe-b7be-06b363ed9f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:14.640932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b7633-c94d-4cbe-b7be-06b363ed9f36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:14.641423 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b7633-c94d-4cbe-b7be-06b363ed9f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:14.643706 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b7633-c94d-4cbe-b7be-06b363ed9f36 admin admin] PATH_INFO: `/v3/projects/100c2c9e36264dd995b1597899de33aa` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:14.648239 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25b7633-c94d-4cbe-b7be-06b363ed9f36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=100c2c9e36264dd995b1597899de33aa)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:14.652554 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25b7633-c94d-4cbe-b7be-06b363ed9f36 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:14.714802 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-9dad6a50-1f7c-4bc7-a60d-4b3762c67bd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bccc002c73847b3b688c5eee1cef8b0', 'request_id': 'req-9dad6a50-1f7c-4bc7-a60d-4b3762c67bd6'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:14.738361 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-9dad6a50-1f7c-4bc7-a60d-4b3762c67bd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bccc002c73847b3b688c5eee1cef8b0', 'request_id': 'req-9dad6a50-1f7c-4bc7-a60d-4b3762c67bd6'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:14.763550 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-9dad6a50-1f7c-4bc7-a60d-4b3762c67bd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bccc002c73847b3b688c5eee1cef8b0', 'request_id': 'req-9dad6a50-1f7c-4bc7-a60d-4b3762c67bd6'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:14.768342 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:14.806632 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-9dad6a50-1f7c-4bc7-a60d-4b3762c67bd6 admin admin] The token cache is being invalidate because project 7bccc002c73847b3b688c5eee1cef8b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:52:14.808760 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-9dad6a50-1f7c-4bc7-a60d-4b3762c67bd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:14.812850 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-9dad6a50-1f7c-4bc7-a60d-4b3762c67bd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:14.820304 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 904/1830] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:52:14 2022] DELETE /identity/v3/projects/7bccc002c73847b3b688c5eee1cef8b0 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:14.825000 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad6a50-1f7c-4bc7-a60d-4b3762c67bd6 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:14.829548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9dad6a50-1f7c-4bc7-a60d-4b3762c67bd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:14.924961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e25b7633-c94d-4cbe-b7be-06b363ed9f36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '100c2c9e36264dd995b1597899de33aa', 'request_id': 'req-e25b7633-c94d-4cbe-b7be-06b363ed9f36'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:14.934780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e25b7633-c94d-4cbe-b7be-06b363ed9f36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '100c2c9e36264dd995b1597899de33aa', 'request_id': 'req-e25b7633-c94d-4cbe-b7be-06b363ed9f36'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:14.956845 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e25b7633-c94d-4cbe-b7be-06b363ed9f36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '100c2c9e36264dd995b1597899de33aa', 'request_id': 'req-e25b7633-c94d-4cbe-b7be-06b363ed9f36'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:15.023670 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e25b7633-c94d-4cbe-b7be-06b363ed9f36 admin admin] The token cache is being invalidate because project 100c2c9e36264dd995b1597899de33aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:52:15.025705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e25b7633-c94d-4cbe-b7be-06b363ed9f36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:15.036031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e25b7633-c94d-4cbe-b7be-06b363ed9f36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:15.046983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 927/1831] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:52:14 2022] DELETE /identity/v3/projects/100c2c9e36264dd995b1597899de33aa => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:15.051547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b7633-c94d-4cbe-b7be-06b363ed9f36 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:15.058107 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e25b7633-c94d-4cbe-b7be-06b363ed9f36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:15.075496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad6a50-1f7c-4bc7-a60d-4b3762c67bd6 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:15.352453 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b7633-c94d-4cbe-b7be-06b363ed9f36 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:15.390194 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73904dc9-ab51-47c2-9cf8-19849a52254e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:15.391078 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73904dc9-ab51-47c2-9cf8-19849a52254e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:15.391526 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73904dc9-ab51-47c2-9cf8-19849a52254e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:15.391890 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73904dc9-ab51-47c2-9cf8-19849a52254e admin admin] PATH_INFO: `/v3/users/b63b0e53a7c2490999ea1f6c0a63c74f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:15.392731 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c309b53-3a2d-4145-812f-e30f4039bafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:15.394255 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73904dc9-ab51-47c2-9cf8-19849a52254e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b63b0e53a7c2490999ea1f6c0a63c74f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:15.395284 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c309b53-3a2d-4145-812f-e30f4039bafb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:15.397505 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c309b53-3a2d-4145-812f-e30f4039bafb admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:15.398648 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c309b53-3a2d-4145-812f-e30f4039bafb admin admin] PATH_INFO: `/v3/users/96455864e2a3489dad2de8569e19c119` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:15.399713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73904dc9-ab51-47c2-9cf8-19849a52254e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:15.404847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c309b53-3a2d-4145-812f-e30f4039bafb admin admin] RBAC: Authorizing `identity:delete_user(user_id=96455864e2a3489dad2de8569e19c119)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:15.424004 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c309b53-3a2d-4145-812f-e30f4039bafb admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:15.515416 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-73904dc9-ab51-47c2-9cf8-19849a52254e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b63b0e53a7c2490999ea1f6c0a63c74f'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:15.546202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-73904dc9-ab51-47c2-9cf8-19849a52254e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b63b0e53a7c2490999ea1f6c0a63c74f'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:15.579899 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-73904dc9-ab51-47c2-9cf8-19849a52254e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b63b0e53a7c2490999ea1f6c0a63c74f'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:15.584571 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-73904dc9-ab51-47c2-9cf8-19849a52254e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b63b0e53a7c2490999ea1f6c0a63c74f'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:15.584571 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-73904dc9-ab51-47c2-9cf8-19849a52254e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b63b0e53a7c2490999ea1f6c0a63c74f'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:15.648780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0c309b53-3a2d-4145-812f-e30f4039bafb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96455864e2a3489dad2de8569e19c119'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:15.679210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0c309b53-3a2d-4145-812f-e30f4039bafb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96455864e2a3489dad2de8569e19c119'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:15.680266 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 905/1832] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:52:14 2022] DELETE /identity/v3/users/b63b0e53a7c2490999ea1f6c0a63c74f => generated 0 bytes in 852 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:15.680608 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:15.694162 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0c309b53-3a2d-4145-812f-e30f4039bafb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96455864e2a3489dad2de8569e19c119'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:15.713627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0c309b53-3a2d-4145-812f-e30f4039bafb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96455864e2a3489dad2de8569e19c119'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:15.715146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0c309b53-3a2d-4145-812f-e30f4039bafb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96455864e2a3489dad2de8569e19c119'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:15.743128 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 928/1833] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:52:15 2022] DELETE /identity/v3/users/96455864e2a3489dad2de8569e19c119 => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:17.831123 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73904dc9-ab51-47c2-9cf8-19849a52254e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:17.832842 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-73904dc9-ab51-47c2-9cf8-19849a52254e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:17.884419 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:18.019246 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73904dc9-ab51-47c2-9cf8-19849a52254e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:18.108989 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc3b82-7d57-4b7f-bfee-eb1c2c7f0302 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:18.108989 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc3b82-7d57-4b7f-bfee-eb1c2c7f0302 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:18.108989 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc3b82-7d57-4b7f-bfee-eb1c2c7f0302 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:18.108989 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 929/1834] 10.222.0.71 () {54 vars in 921 bytes} [Fri Aug 26 08:52:18 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:52:18.129014 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d4ea9-b55e-459d-80ae-cffebda17fca None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:18.129301 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d4ea9-b55e-459d-80ae-cffebda17fca None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:18.129591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d4ea9-b55e-459d-80ae-cffebda17fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:18.177306 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-dd4d4ea9-b55e-459d-80ae-cffebda17fca None None] MFA Rules not processed for user `f16d1768715a41648af3fe6afadd7477`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:18.178159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40adcd12-01ae-408b-8c3e-210589245a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:18.180705 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adcd12-01ae-408b-8c3e-210589245a81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:18.180979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adcd12-01ae-408b-8c3e-210589245a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:18.181378 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adcd12-01ae-408b-8c3e-210589245a81 admin admin] PATH_INFO: `/v3/users/5d079fd7a2c946f19f5d8dbd446a7882` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:18.182034 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-dd4d4ea9-b55e-459d-80ae-cffebda17fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:18.184407 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40adcd12-01ae-408b-8c3e-210589245a81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d079fd7a2c946f19f5d8dbd446a7882)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:18.190595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40adcd12-01ae-408b-8c3e-210589245a81 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:18.339468 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-40adcd12-01ae-408b-8c3e-210589245a81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d079fd7a2c946f19f5d8dbd446a7882'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:18.371396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-40adcd12-01ae-408b-8c3e-210589245a81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d079fd7a2c946f19f5d8dbd446a7882'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:18.422465 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-40adcd12-01ae-408b-8c3e-210589245a81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d079fd7a2c946f19f5d8dbd446a7882'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:18.427806 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-40adcd12-01ae-408b-8c3e-210589245a81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d079fd7a2c946f19f5d8dbd446a7882'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:18.431356 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-40adcd12-01ae-408b-8c3e-210589245a81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d079fd7a2c946f19f5d8dbd446a7882'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:18.477309 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 930/1835] 10.222.0.71 () {58 vars in 1018 bytes} [Fri Aug 26 08:52:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:18.478782 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d4ea9-b55e-459d-80ae-cffebda17fca None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:18.481520 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-dd4d4ea9-b55e-459d-80ae-cffebda17fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:18.498860 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 906/1836] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:52:17 2022] DELETE /identity/v3/users/5d079fd7a2c946f19f5d8dbd446a7882 => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:18.504062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ba44a-e590-4bd8-8c95-5e9fdcc4be77 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:18.504620 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ba44a-e590-4bd8-8c95-5e9fdcc4be77 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:18.505053 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ba44a-e590-4bd8-8c95-5e9fdcc4be77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:18.515093 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d4ea9-b55e-459d-80ae-cffebda17fca None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:18.555175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c86ba44a-e590-4bd8-8c95-5e9fdcc4be77 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:18.557013 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c86ba44a-e590-4bd8-8c95-5e9fdcc4be77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:18.689210 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:18.718926 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf043dde-edef-4911-a44d-864f4830c3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:18.718926 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf043dde-edef-4911-a44d-864f4830c3d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:18.718926 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf043dde-edef-4911-a44d-864f4830c3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:18.718926 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf043dde-edef-4911-a44d-864f4830c3d6 admin admin] PATH_INFO: `/v3/projects/a31c296151c34140ab45ec45754ef9c3` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:18.720441 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf043dde-edef-4911-a44d-864f4830c3d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a31c296151c34140ab45ec45754ef9c3)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:18.723588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf043dde-edef-4911-a44d-864f4830c3d6 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:18.743529 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 907/1837] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:52:18.745955 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ba44a-e590-4bd8-8c95-5e9fdcc4be77 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:18.750372 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c86ba44a-e590-4bd8-8c95-5e9fdcc4be77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:18.782522 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ba44a-e590-4bd8-8c95-5e9fdcc4be77 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:18.788881 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0190e03-c392-481d-8489-807c9ea6d629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:18.789630 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0190e03-c392-481d-8489-807c9ea6d629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:18.789909 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0190e03-c392-481d-8489-807c9ea6d629 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:18.790187 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0190e03-c392-481d-8489-807c9ea6d629 admin admin] PATH_INFO: `/v3/projects/b1b60eab32a94271977129bf7af6b1b7` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:18.792215 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0190e03-c392-481d-8489-807c9ea6d629 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1b60eab32a94271977129bf7af6b1b7)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:18.794019 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0190e03-c392-481d-8489-807c9ea6d629 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:18.915148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cf043dde-edef-4911-a44d-864f4830c3d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a31c296151c34140ab45ec45754ef9c3', 'request_id': 'req-cf043dde-edef-4911-a44d-864f4830c3d6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:18.933819 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cf043dde-edef-4911-a44d-864f4830c3d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a31c296151c34140ab45ec45754ef9c3', 'request_id': 'req-cf043dde-edef-4911-a44d-864f4830c3d6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:18.953502 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e0190e03-c392-481d-8489-807c9ea6d629 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1b60eab32a94271977129bf7af6b1b7', 'request_id': 'req-e0190e03-c392-481d-8489-807c9ea6d629'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:18.956456 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cf043dde-edef-4911-a44d-864f4830c3d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a31c296151c34140ab45ec45754ef9c3', 'request_id': 'req-cf043dde-edef-4911-a44d-864f4830c3d6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:18.969519 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e0190e03-c392-481d-8489-807c9ea6d629 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1b60eab32a94271977129bf7af6b1b7', 'request_id': 'req-e0190e03-c392-481d-8489-807c9ea6d629'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:18.980191 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e0190e03-c392-481d-8489-807c9ea6d629 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1b60eab32a94271977129bf7af6b1b7', 'request_id': 'req-e0190e03-c392-481d-8489-807c9ea6d629'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:18.997585 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cf043dde-edef-4911-a44d-864f4830c3d6 admin admin] The token cache is being invalidate because project a31c296151c34140ab45ec45754ef9c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:52:18.998248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cf043dde-edef-4911-a44d-864f4830c3d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:18.999256 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cf043dde-edef-4911-a44d-864f4830c3d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:19.001943 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 931/1838] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:52:18 2022] DELETE /identity/v3/projects/a31c296151c34140ab45ec45754ef9c3 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:19.003461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf043dde-edef-4911-a44d-864f4830c3d6 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:19.009593 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-cf043dde-edef-4911-a44d-864f4830c3d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:19.011994 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e0190e03-c392-481d-8489-807c9ea6d629 admin admin] The token cache is being invalidate because project b1b60eab32a94271977129bf7af6b1b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:52:19.012397 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e0190e03-c392-481d-8489-807c9ea6d629 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:19.013148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e0190e03-c392-481d-8489-807c9ea6d629 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:19.014777 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 908/1839] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:52:18 2022] DELETE /identity/v3/projects/b1b60eab32a94271977129bf7af6b1b7 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:19.020469 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0190e03-c392-481d-8489-807c9ea6d629 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:19.020469 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e0190e03-c392-481d-8489-807c9ea6d629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:19.026694 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:19.169435 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf043dde-edef-4911-a44d-864f4830c3d6 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:19.184813 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0190e03-c392-481d-8489-807c9ea6d629 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:19.201569 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97e490-9fd7-4211-b464-9b83ea30db76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:19.202758 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97e490-9fd7-4211-b464-9b83ea30db76 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:19.209832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97e490-9fd7-4211-b464-9b83ea30db76 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:19.211332 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97e490-9fd7-4211-b464-9b83ea30db76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:19.220577 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-eb97e490-9fd7-4211-b464-9b83ea30db76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:19.273860 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8c912-abf2-492e-a3cd-63d55fef3c51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:19.274877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8c912-abf2-492e-a3cd-63d55fef3c51 service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:19.275405 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8c912-abf2-492e-a3cd-63d55fef3c51 service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:19.275916 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8c912-abf2-492e-a3cd-63d55fef3c51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:19.278846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-10a8c912-abf2-492e-a3cd-63d55fef3c51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:19.295823 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8c912-abf2-492e-a3cd-63d55fef3c51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:19.298562 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8c912-abf2-492e-a3cd-63d55fef3c51 service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:19.320241 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 932/1840] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:19.321661 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8c912-abf2-492e-a3cd-63d55fef3c51 service nova] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:19.324433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-10a8c912-abf2-492e-a3cd-63d55fef3c51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:19.458115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8c912-abf2-492e-a3cd-63d55fef3c51 service nova] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:19.467216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78639e71-8e29-4612-a486-0162029966e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:19.468405 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78639e71-8e29-4612-a486-0162029966e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:19.470001 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78639e71-8e29-4612-a486-0162029966e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:19.470463 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78639e71-8e29-4612-a486-0162029966e6 admin admin] PATH_INFO: `/v3/users/ba08c216301746d2be55a2759d763897` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:19.473299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78639e71-8e29-4612-a486-0162029966e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba08c216301746d2be55a2759d763897)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:19.478542 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78639e71-8e29-4612-a486-0162029966e6 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:19.563143 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb97e490-9fd7-4211-b464-9b83ea30db76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:19.564668 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:19.574217 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb97e490-9fd7-4211-b464-9b83ea30db76 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:19.591446 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-78639e71-8e29-4612-a486-0162029966e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba08c216301746d2be55a2759d763897'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:19.613924 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-78639e71-8e29-4612-a486-0162029966e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba08c216301746d2be55a2759d763897'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:19.614645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 909/1841] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:19.618955 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9332c3-e2e3-4168-8007-339e37022820 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:19.634461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-78639e71-8e29-4612-a486-0162029966e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba08c216301746d2be55a2759d763897'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:19.636683 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9332c3-e2e3-4168-8007-339e37022820 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:19.637509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9332c3-e2e3-4168-8007-339e37022820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:19.654013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-78639e71-8e29-4612-a486-0162029966e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba08c216301746d2be55a2759d763897'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:19.656356 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-78639e71-8e29-4612-a486-0162029966e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba08c216301746d2be55a2759d763897'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:19.670133 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 933/1842] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:52:19 2022] DELETE /identity/v3/users/ba08c216301746d2be55a2759d763897 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:19.718118 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-5c9332c3-e2e3-4168-8007-339e37022820 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:19.723578 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5c9332c3-e2e3-4168-8007-339e37022820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:19.772784 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:19.910621 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 910/1843] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:19.923133 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186bbde-bc81-45f1-80a3-c4e115ac8b30 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:19.923514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186bbde-bc81-45f1-80a3-c4e115ac8b30 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:19.923862 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186bbde-bc81-45f1-80a3-c4e115ac8b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:19.976365 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-3186bbde-bc81-45f1-80a3-c4e115ac8b30 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:19.978685 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3186bbde-bc81-45f1-80a3-c4e115ac8b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:20.000319 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 934/1844] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:52:20.003578 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9332c3-e2e3-4168-8007-339e37022820 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:20.019651 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9332c3-e2e3-4168-8007-339e37022820 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:20.031275 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfde332-8e2f-447b-80c3-44ec3655c0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:20.033831 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfde332-8e2f-447b-80c3-44ec3655c0b9 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:20.039367 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfde332-8e2f-447b-80c3-44ec3655c0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:20.039970 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfde332-8e2f-447b-80c3-44ec3655c0b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:20.040649 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfde332-8e2f-447b-80c3-44ec3655c0b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:52:20.041041 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfde332-8e2f-447b-80c3-44ec3655c0b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:20.043057 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfde332-8e2f-447b-80c3-44ec3655c0b9 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:20.079793 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 911/1845] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:52:20.080907 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186bbde-bc81-45f1-80a3-c4e115ac8b30 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:20.111569 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186bbde-bc81-45f1-80a3-c4e115ac8b30 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:20.148854 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782214f8-c7b5-4676-ae95-3dcd267f6c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:20.150024 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782214f8-c7b5-4676-ae95-3dcd267f6c25 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:20.150504 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782214f8-c7b5-4676-ae95-3dcd267f6c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:20.152173 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782214f8-c7b5-4676-ae95-3dcd267f6c25 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:20.153497 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782214f8-c7b5-4676-ae95-3dcd267f6c25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:20.175844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782214f8-c7b5-4676-ae95-3dcd267f6c25 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:20.286181 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 935/1846] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:52:20 2022] POST /identity/v3/projects => generated 363 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:20.288389 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782214f8-c7b5-4676-ae95-3dcd267f6c25 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:20.288616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfde332-8e2f-447b-80c3-44ec3655c0b9 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:20.292252 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-acfde332-8e2f-447b-80c3-44ec3655c0b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:20.332774 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfde332-8e2f-447b-80c3-44ec3655c0b9 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:20.516230 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782214f8-c7b5-4676-ae95-3dcd267f6c25 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:20.532174 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c995517-d5ad-4b0f-bc6b-e964587f09c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:20.533099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c995517-d5ad-4b0f-bc6b-e964587f09c9 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:20.533474 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c995517-d5ad-4b0f-bc6b-e964587f09c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:20.533865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c995517-d5ad-4b0f-bc6b-e964587f09c9 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:20.534907 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c995517-d5ad-4b0f-bc6b-e964587f09c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:20.542151 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c995517-d5ad-4b0f-bc6b-e964587f09c9 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:20.614999 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c88e1-1dd0-4fd1-893e-5d972162ed9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:20.618387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c88e1-1dd0-4fd1-893e-5d972162ed9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:20.618707 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c88e1-1dd0-4fd1-893e-5d972162ed9b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:20.619095 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c88e1-1dd0-4fd1-893e-5d972162ed9b admin admin] PATH_INFO: `/v3/projects/081e0916084e42a58615501194d6364e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:20.622713 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 936/1847] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:52:20 2022] POST /identity/v3/users => generated 282 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:20.630673 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c88e1-1dd0-4fd1-893e-5d972162ed9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=081e0916084e42a58615501194d6364e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:20.632556 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c995517-d5ad-4b0f-bc6b-e964587f09c9 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:20.637037 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c88e1-1dd0-4fd1-893e-5d972162ed9b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:20.674304 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c995517-d5ad-4b0f-bc6b-e964587f09c9 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:20.691434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147cb71-5928-4743-881e-3ad2c080fba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:20.695184 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147cb71-5928-4743-881e-3ad2c080fba3 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:20.697198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147cb71-5928-4743-881e-3ad2c080fba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:20.700432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147cb71-5928-4743-881e-3ad2c080fba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:20.704321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147cb71-5928-4743-881e-3ad2c080fba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:20.710375 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147cb71-5928-4743-881e-3ad2c080fba3 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:20.751005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 937/1848] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:20.752021 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147cb71-5928-4743-881e-3ad2c080fba3 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:20.787126 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147cb71-5928-4743-881e-3ad2c080fba3 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:20.822359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0e9c88e1-1dd0-4fd1-893e-5d972162ed9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '081e0916084e42a58615501194d6364e', 'request_id': 'req-0e9c88e1-1dd0-4fd1-893e-5d972162ed9b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:20.843668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0e9c88e1-1dd0-4fd1-893e-5d972162ed9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '081e0916084e42a58615501194d6364e', 'request_id': 'req-0e9c88e1-1dd0-4fd1-893e-5d972162ed9b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:20.853980 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0e9c88e1-1dd0-4fd1-893e-5d972162ed9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '081e0916084e42a58615501194d6364e', 'request_id': 'req-0e9c88e1-1dd0-4fd1-893e-5d972162ed9b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:20.879131 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0e9c88e1-1dd0-4fd1-893e-5d972162ed9b admin admin] The token cache is being invalidate because project 081e0916084e42a58615501194d6364e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:52:20.879862 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0e9c88e1-1dd0-4fd1-893e-5d972162ed9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:20.881187 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0e9c88e1-1dd0-4fd1-893e-5d972162ed9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:20.888372 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 912/1849] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:52:20 2022] DELETE /identity/v3/projects/081e0916084e42a58615501194d6364e => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:20.893011 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c7f35-a328-47a7-9b67-4703e1516030 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:20.893603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c7f35-a328-47a7-9b67-4703e1516030 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:20.894279 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c7f35-a328-47a7-9b67-4703e1516030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:20.943816 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c71c7f35-a328-47a7-9b67-4703e1516030 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:20.950527 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c71c7f35-a328-47a7-9b67-4703e1516030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:20.981960 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:21.003287 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b7137-82b8-4b23-8120-f9d025adec30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:21.004498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b7137-82b8-4b23-8120-f9d025adec30 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:21.005216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b7137-82b8-4b23-8120-f9d025adec30 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:21.006020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b7137-82b8-4b23-8120-f9d025adec30 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:21.015823 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550b7137-82b8-4b23-8120-f9d025adec30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:21.025769 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550b7137-82b8-4b23-8120-f9d025adec30 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:21.081271 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 938/1850] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:21.085440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b7137-82b8-4b23-8120-f9d025adec30 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:21.088063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-550b7137-82b8-4b23-8120-f9d025adec30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:21.125023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b7137-82b8-4b23-8120-f9d025adec30 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:21.165457 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebabac0-0446-4258-b3a0-43aeca7065a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:21.169002 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:21.170755 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebabac0-0446-4258-b3a0-43aeca7065a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:21.174651 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebabac0-0446-4258-b3a0-43aeca7065a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:21.175033 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebabac0-0446-4258-b3a0-43aeca7065a5 admin admin] PATH_INFO: `/v3/projects/a3d411aca75d49989de42d3e907f25e9/users/c74fce0dff824bdc8e8013278d8eeb80/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:21.231773 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebabac0-0446-4258-b3a0-43aeca7065a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3d411aca75d49989de42d3e907f25e9, user_id=c74fce0dff824bdc8e8013278d8eeb80, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:21.234676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebabac0-0446-4258-b3a0-43aeca7065a5 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:21.265371 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 939/1851] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:21 2022] PUT /identity/v3/projects/a3d411aca75d49989de42d3e907f25e9/users/c74fce0dff824bdc8e8013278d8eeb80/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:52:21.275673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6b15c-ab07-4012-878c-1d66d4179fbf None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:21.275960 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6b15c-ab07-4012-878c-1d66d4179fbf None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:21.276240 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6b15c-ab07-4012-878c-1d66d4179fbf None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:21.281065 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 913/1852] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:21.281536 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ea44a-92a2-475c-9ff2-0aec9c0579e2 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:21.290159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ea44a-92a2-475c-9ff2-0aec9c0579e2 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:21.290159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ea44a-92a2-475c-9ff2-0aec9c0579e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:21.291232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 940/1853] 10.222.0.71 () {54 vars in 921 bytes} [Fri Aug 26 08:52:21 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:52:21.292218 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f6e2d-0802-458d-baf7-e1c71ae8d1c4 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:21.292608 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f6e2d-0802-458d-baf7-e1c71ae8d1c4 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:21.294526 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f6e2d-0802-458d-baf7-e1c71ae8d1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:21.344349 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-402f6e2d-0802-458d-baf7-e1c71ae8d1c4 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:21.344953 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-402f6e2d-0802-458d-baf7-e1c71ae8d1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:21.370153 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-0b6ea44a-92a2-475c-9ff2-0aec9c0579e2 None None] MFA Rules not processed for user `c74fce0dff824bdc8e8013278d8eeb80`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:21.371594 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0b6ea44a-92a2-475c-9ff2-0aec9c0579e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:21.483045 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 941/1854] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:21.484523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07f0b6-e423-4896-9d5b-a2135cbaa5c9 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:21.484817 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07f0b6-e423-4896-9d5b-a2135cbaa5c9 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:21.485167 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07f0b6-e423-4896-9d5b-a2135cbaa5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:21.517479 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-cf07f0b6-e423-4896-9d5b-a2135cbaa5c9 None None] MFA Rules not processed for user `f16d1768715a41648af3fe6afadd7477`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:21.519236 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-cf07f0b6-e423-4896-9d5b-a2135cbaa5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:21.525247 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 914/1855] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:21.525247 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ea44a-92a2-475c-9ff2-0aec9c0579e2 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:21.540962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ea44a-92a2-475c-9ff2-0aec9c0579e2 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:21.563335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585bfab-f0a8-4fbd-9940-51b651760f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:21.564204 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585bfab-f0a8-4fbd-9940-51b651760f9e admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:21.564639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585bfab-f0a8-4fbd-9940-51b651760f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:21.564999 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585bfab-f0a8-4fbd-9940-51b651760f9e admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:21.566040 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585bfab-f0a8-4fbd-9940-51b651760f9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:52:21.566524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585bfab-f0a8-4fbd-9940-51b651760f9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:21.568843 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585bfab-f0a8-4fbd-9940-51b651760f9e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:21.584842 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 915/1856] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:52:21.585825 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585bfab-f0a8-4fbd-9940-51b651760f9e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:21.609571 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585bfab-f0a8-4fbd-9940-51b651760f9e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:21.622323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94113e2e-d9da-496f-aa23-af7e5a96c189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:21.623411 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94113e2e-d9da-496f-aa23-af7e5a96c189 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:21.624573 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94113e2e-d9da-496f-aa23-af7e5a96c189 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:21.625290 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94113e2e-d9da-496f-aa23-af7e5a96c189 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:21.626412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94113e2e-d9da-496f-aa23-af7e5a96c189 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:21.629891 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94113e2e-d9da-496f-aa23-af7e5a96c189 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:21.635919 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 942/1857] 10.222.0.71 () {58 vars in 1018 bytes} [Fri Aug 26 08:52:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:21.637284 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07f0b6-e423-4896-9d5b-a2135cbaa5c9 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:21.667354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07f0b6-e423-4896-9d5b-a2135cbaa5c9 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:21.678037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6ef55-b187-4471-8252-2bda3fe7d786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:21.678700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6ef55-b187-4471-8252-2bda3fe7d786 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:21.679023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6ef55-b187-4471-8252-2bda3fe7d786 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:21.679309 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6ef55-b187-4471-8252-2bda3fe7d786 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:21.680401 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6ef55-b187-4471-8252-2bda3fe7d786 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:21.682309 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6ef55-b187-4471-8252-2bda3fe7d786 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:21.721378 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 916/1858] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:52:21 2022] POST /identity/v3/projects => generated 363 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:21.743831 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94113e2e-d9da-496f-aa23-af7e5a96c189 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:21.745160 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 943/1859] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:52:21 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:21.746613 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6ef55-b187-4471-8252-2bda3fe7d786 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:21.749790 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-94113e2e-d9da-496f-aa23-af7e5a96c189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:21.825607 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:21.856788 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6ef55-b187-4471-8252-2bda3fe7d786 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:21.863028 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c69f3-975a-4b64-ac4b-446c9c2bc851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:21.863696 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c69f3-975a-4b64-ac4b-446c9c2bc851 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:21.864049 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c69f3-975a-4b64-ac4b-446c9c2bc851 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:21.864325 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c69f3-975a-4b64-ac4b-446c9c2bc851 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:21.865099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c69f3-975a-4b64-ac4b-446c9c2bc851 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:21.868142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c69f3-975a-4b64-ac4b-446c9c2bc851 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:21.916252 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 944/1860] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:52:21 2022] POST /identity/v3/users => generated 288 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:21.918370 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c69f3-975a-4b64-ac4b-446c9c2bc851 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:21.938941 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c69f3-975a-4b64-ac4b-446c9c2bc851 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:21.946539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86669daf-a583-4e74-943a-92c1729221be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:21.947761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86669daf-a583-4e74-943a-92c1729221be admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:21.948055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86669daf-a583-4e74-943a-92c1729221be admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:21.948335 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86669daf-a583-4e74-943a-92c1729221be admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:21.949191 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86669daf-a583-4e74-943a-92c1729221be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:21.951849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86669daf-a583-4e74-943a-92c1729221be admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:21.957274 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94113e2e-d9da-496f-aa23-af7e5a96c189 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:22.000631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 945/1861] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:52:21 2022] POST /identity/v3/users => generated 288 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:22.001144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86669daf-a583-4e74-943a-92c1729221be admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:22.011930 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:22.028292 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86669daf-a583-4e74-943a-92c1729221be admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:22.036583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1e97f-d17e-40bd-b778-fee4ee6b50b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:22.037369 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1e97f-d17e-40bd-b778-fee4ee6b50b5 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:22.037656 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1e97f-d17e-40bd-b778-fee4ee6b50b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:22.037949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1e97f-d17e-40bd-b778-fee4ee6b50b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:22.038703 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e1e97f-d17e-40bd-b778-fee4ee6b50b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:22.040920 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e1e97f-d17e-40bd-b778-fee4ee6b50b5 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:22.057777 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 946/1862] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:22.059002 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1e97f-d17e-40bd-b778-fee4ee6b50b5 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:22.069012 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7431d5f-0347-45ed-aaf7-f3a675c0e95d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:22.070093 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7431d5f-0347-45ed-aaf7-f3a675c0e95d service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:22.070878 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7431d5f-0347-45ed-aaf7-f3a675c0e95d service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:22.071420 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7431d5f-0347-45ed-aaf7-f3a675c0e95d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:22.080097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1e97f-d17e-40bd-b778-fee4ee6b50b5 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:22.086194 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7431d5f-0347-45ed-aaf7-f3a675c0e95d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:22.086749 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa0a93-b3e6-40ac-93b9-2b2f215d468e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:22.088609 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa0a93-b3e6-40ac-93b9-2b2f215d468e admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:22.088954 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa0a93-b3e6-40ac-93b9-2b2f215d468e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:22.089338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa0a93-b3e6-40ac-93b9-2b2f215d468e admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:22.090501 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7431d5f-0347-45ed-aaf7-f3a675c0e95d service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:22.090749 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aa0a93-b3e6-40ac-93b9-2b2f215d468e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:22.093117 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aa0a93-b3e6-40ac-93b9-2b2f215d468e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:22.108911 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 917/1863] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:22.112313 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7431d5f-0347-45ed-aaf7-f3a675c0e95d service nova] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:22.131468 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 947/1864] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:22.135010 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa0a93-b3e6-40ac-93b9-2b2f215d468e admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:22.136679 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7431d5f-0347-45ed-aaf7-f3a675c0e95d service nova] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:22.151168 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336aaf2-1d4b-40a6-beb1-b858145f5e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:22.152364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336aaf2-1d4b-40a6-beb1-b858145f5e70 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:22.152865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336aaf2-1d4b-40a6-beb1-b858145f5e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:22.153709 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0336aaf2-1d4b-40a6-beb1-b858145f5e70 admin admin] PATH_INFO: `/v3/projects/42e46784f64441c99a3fa30526655d5a/users/db512ee6de9944669d9eaf2b5ae610e7/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:22.173646 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa0a93-b3e6-40ac-93b9-2b2f215d468e admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:22.189345 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0336aaf2-1d4b-40a6-beb1-b858145f5e70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42e46784f64441c99a3fa30526655d5a, user_id=db512ee6de9944669d9eaf2b5ae610e7, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:22.193002 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab48e2-f72d-42c5-88e3-ce88dabebbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:22.194309 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab48e2-f72d-42c5-88e3-ce88dabebbf6 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:22.195625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab48e2-f72d-42c5-88e3-ce88dabebbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:22.198616 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab48e2-f72d-42c5-88e3-ce88dabebbf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:22.199645 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab48e2-f72d-42c5-88e3-ce88dabebbf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:22.200882 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0336aaf2-1d4b-40a6-beb1-b858145f5e70 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:22.205224 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab48e2-f72d-42c5-88e3-ce88dabebbf6 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:22.255390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 948/1865] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:22.257933 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab48e2-f72d-42c5-88e3-ce88dabebbf6 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:22.285007 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab48e2-f72d-42c5-88e3-ce88dabebbf6 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:22.310959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 918/1866] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:22 2022] PUT /identity/v3/projects/42e46784f64441c99a3fa30526655d5a/users/db512ee6de9944669d9eaf2b5ae610e7/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:22.318005 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336aaf2-1d4b-40a6-beb1-b858145f5e70 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:22.421163 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7dc7e-3c2f-496a-838a-6f2392c376a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:22.422203 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7dc7e-3c2f-496a-838a-6f2392c376a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:22.422502 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7dc7e-3c2f-496a-838a-6f2392c376a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:22.422799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7dc7e-3c2f-496a-838a-6f2392c376a8 admin admin] PATH_INFO: `/v3/projects/4781b11c08544fb481a5126565bb9bee/users/f7ba7907de034abfa0a85cff36821812/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:22.448090 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb7dc7e-3c2f-496a-838a-6f2392c376a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4781b11c08544fb481a5126565bb9bee, user_id=f7ba7907de034abfa0a85cff36821812, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:22.452350 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb7dc7e-3c2f-496a-838a-6f2392c376a8 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:22.464671 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0336aaf2-1d4b-40a6-beb1-b858145f5e70 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:22.479811 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 949/1867] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:22 2022] PUT /identity/v3/projects/4781b11c08544fb481a5126565bb9bee/users/f7ba7907de034abfa0a85cff36821812/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:22.482982 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7dc7e-3c2f-496a-838a-6f2392c376a8 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:22.499815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4fb7dc7e-3c2f-496a-838a-6f2392c376a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:22.676515 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7dc7e-3c2f-496a-838a-6f2392c376a8 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:22.683785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ab0176-e433-4140-986c-47267ba332ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:22.684677 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab0176-e433-4140-986c-47267ba332ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:22.685000 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab0176-e433-4140-986c-47267ba332ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:22.685286 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab0176-e433-4140-986c-47267ba332ad admin admin] PATH_INFO: `/v3/projects/a63163b33b2c4f649ace39b8e22eb8bf` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:22.688122 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ab0176-e433-4140-986c-47267ba332ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=a63163b33b2c4f649ace39b8e22eb8bf)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:22.691320 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ab0176-e433-4140-986c-47267ba332ad admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:22.704809 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93038236-bc6c-476f-b6e2-d42b67771244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:22.708012 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93038236-bc6c-476f-b6e2-d42b67771244 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:22.708012 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93038236-bc6c-476f-b6e2-d42b67771244 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:22.708181 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93038236-bc6c-476f-b6e2-d42b67771244 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:22.711307 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93038236-bc6c-476f-b6e2-d42b67771244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:22.714680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:22.742802 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93038236-bc6c-476f-b6e2-d42b67771244 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:22.784719 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 919/1868] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 469 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:22.787653 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93038236-bc6c-476f-b6e2-d42b67771244 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:22.795930 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-93038236-bc6c-476f-b6e2-d42b67771244 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:22.844807 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93038236-bc6c-476f-b6e2-d42b67771244 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:22.864094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-98ab0176-e433-4140-986c-47267ba332ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a63163b33b2c4f649ace39b8e22eb8bf', 'request_id': 'req-98ab0176-e433-4140-986c-47267ba332ad'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:22.887657 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-98ab0176-e433-4140-986c-47267ba332ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a63163b33b2c4f649ace39b8e22eb8bf', 'request_id': 'req-98ab0176-e433-4140-986c-47267ba332ad'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:22.907838 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-98ab0176-e433-4140-986c-47267ba332ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a63163b33b2c4f649ace39b8e22eb8bf', 'request_id': 'req-98ab0176-e433-4140-986c-47267ba332ad'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:22.937016 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-98ab0176-e433-4140-986c-47267ba332ad admin admin] The token cache is being invalidate because project a63163b33b2c4f649ace39b8e22eb8bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:52:22.937627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-98ab0176-e433-4140-986c-47267ba332ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:22.938858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-98ab0176-e433-4140-986c-47267ba332ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:22.943324 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 950/1869] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:52:22 2022] DELETE /identity/v3/projects/a63163b33b2c4f649ace39b8e22eb8bf => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:22.947296 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:22.972580 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec952dec-1642-4ad4-a565-bb394414fc99 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:22.973194 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec952dec-1642-4ad4-a565-bb394414fc99 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:22.973740 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec952dec-1642-4ad4-a565-bb394414fc99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:23.012235 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771580a-e18a-4081-bdf4-1b0b12822fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:23.013855 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771580a-e18a-4081-bdf4-1b0b12822fe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:23.014478 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771580a-e18a-4081-bdf4-1b0b12822fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:23.015135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771580a-e18a-4081-bdf4-1b0b12822fe7 admin admin] PATH_INFO: `/v3/projects/82de73713cad4cd9ace2db1a039d5398` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:23.018423 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771580a-e18a-4081-bdf4-1b0b12822fe7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82de73713cad4cd9ace2db1a039d5398)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:23.023391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771580a-e18a-4081-bdf4-1b0b12822fe7 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:23.028881 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ec952dec-1642-4ad4-a565-bb394414fc99 None None] MFA Rules not processed for user `f7ba7907de034abfa0a85cff36821812`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:23.030310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ec952dec-1642-4ad4-a565-bb394414fc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:23.147060 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a771580a-e18a-4081-bdf4-1b0b12822fe7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82de73713cad4cd9ace2db1a039d5398', 'request_id': 'req-a771580a-e18a-4081-bdf4-1b0b12822fe7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:23.165009 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a771580a-e18a-4081-bdf4-1b0b12822fe7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82de73713cad4cd9ace2db1a039d5398', 'request_id': 'req-a771580a-e18a-4081-bdf4-1b0b12822fe7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:23.177441 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a771580a-e18a-4081-bdf4-1b0b12822fe7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82de73713cad4cd9ace2db1a039d5398', 'request_id': 'req-a771580a-e18a-4081-bdf4-1b0b12822fe7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:23.204018 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a771580a-e18a-4081-bdf4-1b0b12822fe7 admin admin] The token cache is being invalidate because project 82de73713cad4cd9ace2db1a039d5398 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:52:23.204652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a771580a-e18a-4081-bdf4-1b0b12822fe7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:23.205137 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a771580a-e18a-4081-bdf4-1b0b12822fe7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:23.207461 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 920/1870] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:52:22 2022] DELETE /identity/v3/projects/82de73713cad4cd9ace2db1a039d5398 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:23.208636 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771580a-e18a-4081-bdf4-1b0b12822fe7 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:23.210142 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a771580a-e18a-4081-bdf4-1b0b12822fe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:23.237728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 951/1871] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:23.239357 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec952dec-1642-4ad4-a565-bb394414fc99 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:23.242443 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ec952dec-1642-4ad4-a565-bb394414fc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:23.261289 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec952dec-1642-4ad4-a565-bb394414fc99 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:23.327695 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771580a-e18a-4081-bdf4-1b0b12822fe7 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:23.333720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d7817-99e7-4b4e-9f34-385cc291c663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:23.334487 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d7817-99e7-4b4e-9f34-385cc291c663 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:23.335085 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d7817-99e7-4b4e-9f34-385cc291c663 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:23.335367 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d7817-99e7-4b4e-9f34-385cc291c663 admin admin] PATH_INFO: `/v3/projects/42e46784f64441c99a3fa30526655d5a/users/db512ee6de9944669d9eaf2b5ae610e7/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:23.335660 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd405ee-5091-4946-a95f-ff00ee8da92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:23.336856 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd405ee-5091-4946-a95f-ff00ee8da92d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:23.337143 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd405ee-5091-4946-a95f-ff00ee8da92d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:23.337411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd405ee-5091-4946-a95f-ff00ee8da92d admin admin] PATH_INFO: `/v3/users/118e1b1cf4b648beae8ba3c221c640d2` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:23.338005 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d7817-99e7-4b4e-9f34-385cc291c663 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42e46784f64441c99a3fa30526655d5a, user_id=db512ee6de9944669d9eaf2b5ae610e7, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:23.338608 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd405ee-5091-4946-a95f-ff00ee8da92d admin admin] RBAC: Authorizing `identity:delete_user(user_id=118e1b1cf4b648beae8ba3c221c640d2)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:23.340690 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd405ee-5091-4946-a95f-ff00ee8da92d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:23.340880 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d7817-99e7-4b4e-9f34-385cc291c663 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:23.364698 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 921/1872] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:23 2022] PUT /identity/v3/projects/42e46784f64441c99a3fa30526655d5a/users/db512ee6de9944669d9eaf2b5ae610e7/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:23.366712 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f092228-2f8f-4acd-b119-723e4629b45c None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:23.367024 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f092228-2f8f-4acd-b119-723e4629b45c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:23.367303 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f092228-2f8f-4acd-b119-723e4629b45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:23.387845 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:23.400807 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-7f092228-2f8f-4acd-b119-723e4629b45c None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:23.402735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7f092228-2f8f-4acd-b119-723e4629b45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:23.431426 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-3bd405ee-5091-4946-a95f-ff00ee8da92d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '118e1b1cf4b648beae8ba3c221c640d2'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:23.453898 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-3bd405ee-5091-4946-a95f-ff00ee8da92d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '118e1b1cf4b648beae8ba3c221c640d2'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:23.468260 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-3bd405ee-5091-4946-a95f-ff00ee8da92d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '118e1b1cf4b648beae8ba3c221c640d2'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:23.477677 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-3bd405ee-5091-4946-a95f-ff00ee8da92d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '118e1b1cf4b648beae8ba3c221c640d2'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:23.478716 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-3bd405ee-5091-4946-a95f-ff00ee8da92d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '118e1b1cf4b648beae8ba3c221c640d2'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:23.485033 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:23.486743 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 952/1873] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:52:23 2022] DELETE /identity/v3/users/118e1b1cf4b648beae8ba3c221c640d2 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:23.487740 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd405ee-5091-4946-a95f-ff00ee8da92d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:23.489206 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3bd405ee-5091-4946-a95f-ff00ee8da92d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:23.502078 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 922/1874] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:23.509410 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e49e52-25f9-46bc-a832-1c48a5e27111 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:23.510372 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e49e52-25f9-46bc-a832-1c48a5e27111 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:23.510656 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e49e52-25f9-46bc-a832-1c48a5e27111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:23.550354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-45e49e52-25f9-46bc-a832-1c48a5e27111 None None] MFA Rules not processed for user `db512ee6de9944669d9eaf2b5ae610e7`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:23.551795 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-45e49e52-25f9-46bc-a832-1c48a5e27111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:23.611370 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd405ee-5091-4946-a95f-ff00ee8da92d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:23.622079 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0832b-b7ac-4591-a9ee-1ef9abc17e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:23.623472 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0832b-b7ac-4591-a9ee-1ef9abc17e31 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:23.625040 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0832b-b7ac-4591-a9ee-1ef9abc17e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:23.625040 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0832b-b7ac-4591-a9ee-1ef9abc17e31 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:23.625783 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0832b-b7ac-4591-a9ee-1ef9abc17e31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:23.629514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0832b-b7ac-4591-a9ee-1ef9abc17e31 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:23.685972 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 953/1875] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:52:23 2022] POST /identity/v3/projects => generated 377 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:23.685972 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fc18f-8530-4d96-a696-85b8e8b9ed1c None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:23.685972 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fc18f-8530-4d96-a696-85b8e8b9ed1c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:23.685972 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fc18f-8530-4d96-a696-85b8e8b9ed1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:23.715106 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 923/1876] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:23 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:23.717799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-c77fc18f-8530-4d96-a696-85b8e8b9ed1c None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:23.719065 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e49e52-25f9-46bc-a832-1c48a5e27111 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:23.719259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c77fc18f-8530-4d96-a696-85b8e8b9ed1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:23.760846 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e49e52-25f9-46bc-a832-1c48a5e27111 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:23.881016 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 954/1877] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:23.881744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77fc18f-8530-4d96-a696-85b8e8b9ed1c None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:23.883830 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c77fc18f-8530-4d96-a696-85b8e8b9ed1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:23.928659 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9dc77b-fe3c-441c-ad2e-18a85f19ad8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:23.929459 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9dc77b-fe3c-441c-ad2e-18a85f19ad8f admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:23.929746 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9dc77b-fe3c-441c-ad2e-18a85f19ad8f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:23.930027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9dc77b-fe3c-441c-ad2e-18a85f19ad8f admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:23.931488 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9dc77b-fe3c-441c-ad2e-18a85f19ad8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:23.933548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9dc77b-fe3c-441c-ad2e-18a85f19ad8f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:23.959238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:23.971394 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 924/1878] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:52:23 2022] POST /identity/v3/users => generated 289 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:23.972793 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9dc77b-fe3c-441c-ad2e-18a85f19ad8f admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:23.987419 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9dc77b-fe3c-441c-ad2e-18a85f19ad8f admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:23.994655 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e032b3-8fa6-4c39-bb68-6d423ccc31a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:23.995933 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e032b3-8fa6-4c39-bb68-6d423ccc31a2 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:23.996487 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e032b3-8fa6-4c39-bb68-6d423ccc31a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:23.997045 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e032b3-8fa6-4c39-bb68-6d423ccc31a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:23.999050 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e032b3-8fa6-4c39-bb68-6d423ccc31a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:52:23.999789 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e032b3-8fa6-4c39-bb68-6d423ccc31a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:24.002074 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e032b3-8fa6-4c39-bb68-6d423ccc31a2 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:24.028530 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 925/1879] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:52:24.030067 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e032b3-8fa6-4c39-bb68-6d423ccc31a2 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:24.032206 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77fc18f-8530-4d96-a696-85b8e8b9ed1c None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:24.038542 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b9a40-5182-4222-9ab9-df2fdb1d2f33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:24.039805 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b9a40-5182-4222-9ab9-df2fdb1d2f33 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:24.040177 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b9a40-5182-4222-9ab9-df2fdb1d2f33 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:24.040453 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b9a40-5182-4222-9ab9-df2fdb1d2f33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:24.053463 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e032b3-8fa6-4c39-bb68-6d423ccc31a2 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:24.060104 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c551c-6737-48df-b567-ecfd3689758b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:24.061240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c551c-6737-48df-b567-ecfd3689758b admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:24.061983 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c551c-6737-48df-b567-ecfd3689758b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:24.062914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c551c-6737-48df-b567-ecfd3689758b admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:24.064170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c551c-6737-48df-b567-ecfd3689758b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:24.066571 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c551c-6737-48df-b567-ecfd3689758b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:24.087267 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 926/1880] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:24.089600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c551c-6737-48df-b567-ecfd3689758b admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:24.108792 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c551c-6737-48df-b567-ecfd3689758b admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:24.116512 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20cba31-0648-46b0-b6db-99f7981a4ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:24.117529 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cba31-0648-46b0-b6db-99f7981a4ecf admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:24.118016 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cba31-0648-46b0-b6db-99f7981a4ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:24.118652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cba31-0648-46b0-b6db-99f7981a4ecf admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:24.119857 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20cba31-0648-46b0-b6db-99f7981a4ecf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:24.123990 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20cba31-0648-46b0-b6db-99f7981a4ecf admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:24.126095 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039b9a40-5182-4222-9ab9-df2fdb1d2f33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:24.127785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039b9a40-5182-4222-9ab9-df2fdb1d2f33 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:24.145916 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 955/1881] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:23 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:24.147035 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b9a40-5182-4222-9ab9-df2fdb1d2f33 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:24.166322 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b9a40-5182-4222-9ab9-df2fdb1d2f33 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:24.172083 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788d4a6-d8e3-413a-9d69-3ffa1351f071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:24.178395 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788d4a6-d8e3-413a-9d69-3ffa1351f071 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:24.178395 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788d4a6-d8e3-413a-9d69-3ffa1351f071 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:24.178395 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788d4a6-d8e3-413a-9d69-3ffa1351f071 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:24.178395 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788d4a6-d8e3-413a-9d69-3ffa1351f071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:24.178395 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788d4a6-d8e3-413a-9d69-3ffa1351f071 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:24.189475 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 927/1882] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:52:24 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:24.195917 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20cba31-0648-46b0-b6db-99f7981a4ecf admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:24.209808 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 956/1883] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:24.223466 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788d4a6-d8e3-413a-9d69-3ffa1351f071 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:24.254091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788d4a6-d8e3-413a-9d69-3ffa1351f071 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:24.259295 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:24.376324 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20cba31-0648-46b0-b6db-99f7981a4ecf admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:24.420009 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e4fff-ac7a-4b9b-af52-ef5493472444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:24.422677 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e4fff-ac7a-4b9b-af52-ef5493472444 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:24.423700 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e4fff-ac7a-4b9b-af52-ef5493472444 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:24.424330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e4fff-ac7a-4b9b-af52-ef5493472444 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:24.426135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e4fff-ac7a-4b9b-af52-ef5493472444 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:24.429079 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e4fff-ac7a-4b9b-af52-ef5493472444 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:24.461590 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde9b8e-1c3b-4a15-82ba-0195c62fcce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:24.462451 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde9b8e-1c3b-4a15-82ba-0195c62fcce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:24.462878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde9b8e-1c3b-4a15-82ba-0195c62fcce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:24.463213 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde9b8e-1c3b-4a15-82ba-0195c62fcce7 admin admin] PATH_INFO: `/v3/projects/1b1a83e28df941a5b94aab9fafe23cf9/users/20bcc74e24f14c57a2cbd31ebb93db5c/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:24.467524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 928/1884] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:52:24 2022] POST /identity/v3/users => generated 291 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:52:24.469639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e4fff-ac7a-4b9b-af52-ef5493472444 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:24.493260 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde9b8e-1c3b-4a15-82ba-0195c62fcce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b1a83e28df941a5b94aab9fafe23cf9, user_id=20bcc74e24f14c57a2cbd31ebb93db5c, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:24.497028 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde9b8e-1c3b-4a15-82ba-0195c62fcce7 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:24.530938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 957/1885] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:24 2022] PUT /identity/v3/projects/1b1a83e28df941a5b94aab9fafe23cf9/users/20bcc74e24f14c57a2cbd31ebb93db5c/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:52:24.530938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde9b8e-1c3b-4a15-82ba-0195c62fcce7 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:24.650095 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e4fff-ac7a-4b9b-af52-ef5493472444 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:24.709938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:24.735406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9691136-cdac-4150-aa5e-f7d2de57cf08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:24.736627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9691136-cdac-4150-aa5e-f7d2de57cf08 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:24.736913 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9691136-cdac-4150-aa5e-f7d2de57cf08 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:24.737307 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9691136-cdac-4150-aa5e-f7d2de57cf08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:24.740945 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a9691136-cdac-4150-aa5e-f7d2de57cf08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:24.741602 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde9b8e-1c3b-4a15-82ba-0195c62fcce7 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:24.752832 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee28d06-a09a-4113-9c9a-dbb21599f52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:24.753635 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee28d06-a09a-4113-9c9a-dbb21599f52f admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:24.753913 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee28d06-a09a-4113-9c9a-dbb21599f52f admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:24.754236 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee28d06-a09a-4113-9c9a-dbb21599f52f admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:24.755133 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee28d06-a09a-4113-9c9a-dbb21599f52f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:24.757880 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee28d06-a09a-4113-9c9a-dbb21599f52f admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:24.785442 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 958/1886] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:24.785442 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b0dc2-60da-4ee7-859c-7de99cdc9a6e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:24.785442 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b0dc2-60da-4ee7-859c-7de99cdc9a6e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:24.785442 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b0dc2-60da-4ee7-859c-7de99cdc9a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:24.836722 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9691136-cdac-4150-aa5e-f7d2de57cf08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:24.838683 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9691136-cdac-4150-aa5e-f7d2de57cf08 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:24.851952 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 929/1887] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:24.853085 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9691136-cdac-4150-aa5e-f7d2de57cf08 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:24.853646 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-345b0dc2-60da-4ee7-859c-7de99cdc9a6e None None] MFA Rules not processed for user `20bcc74e24f14c57a2cbd31ebb93db5c`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:24.856695 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-345b0dc2-60da-4ee7-859c-7de99cdc9a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:24.865972 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9691136-cdac-4150-aa5e-f7d2de57cf08 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:24.871322 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f34e1-f3b2-499a-ad68-6d01f97dce64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:24.872364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f34e1-f3b2-499a-ad68-6d01f97dce64 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:24.872814 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f34e1-f3b2-499a-ad68-6d01f97dce64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:24.873295 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f34e1-f3b2-499a-ad68-6d01f97dce64 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:24.874088 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0f34e1-f3b2-499a-ad68-6d01f97dce64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:24.875784 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0f34e1-f3b2-499a-ad68-6d01f97dce64 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:24.895380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 930/1888] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:24.896629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f34e1-f3b2-499a-ad68-6d01f97dce64 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:24.919487 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f34e1-f3b2-499a-ad68-6d01f97dce64 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:24.931619 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01abd3c8-0ba1-40d1-a622-6ca06302856e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:24.938292 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01abd3c8-0ba1-40d1-a622-6ca06302856e admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:24.938665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01abd3c8-0ba1-40d1-a622-6ca06302856e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:24.939520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01abd3c8-0ba1-40d1-a622-6ca06302856e admin admin] PATH_INFO: `/v3/projects/381a7b4f9d6f406781c9f3184f9c2491/users/cb1ccbd55efc456eb49604003fcabb8b/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:24.967760 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01abd3c8-0ba1-40d1-a622-6ca06302856e admin admin] RBAC: Authorizing `identity:create_grant(project_id=381a7b4f9d6f406781c9f3184f9c2491, user_id=cb1ccbd55efc456eb49604003fcabb8b, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:24.971784 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01abd3c8-0ba1-40d1-a622-6ca06302856e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:24.976389 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:24.998944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 931/1889] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:24 2022] PUT /identity/v3/projects/381a7b4f9d6f406781c9f3184f9c2491/users/cb1ccbd55efc456eb49604003fcabb8b/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:52:25.004713 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 959/1890] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:25.010639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01abd3c8-0ba1-40d1-a622-6ca06302856e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:25.035017 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06dd7d-ab6c-45ef-a1fd-b8b1520aa689 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:25.042317 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06dd7d-ab6c-45ef-a1fd-b8b1520aa689 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:25.042932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06dd7d-ab6c-45ef-a1fd-b8b1520aa689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:25.149414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-5c06dd7d-ab6c-45ef-a1fd-b8b1520aa689 None None] MFA Rules not processed for user `cb1ccbd55efc456eb49604003fcabb8b`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:25.151496 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-5c06dd7d-ab6c-45ef-a1fd-b8b1520aa689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:25.213776 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01abd3c8-0ba1-40d1-a622-6ca06302856e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:25.228484 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b412d7a-f1f7-46ae-b44c-3c6b987a4d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:25.228929 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b412d7a-f1f7-46ae-b44c-3c6b987a4d91 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:25.229560 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b412d7a-f1f7-46ae-b44c-3c6b987a4d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:25.229560 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b412d7a-f1f7-46ae-b44c-3c6b987a4d91 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:25.230116 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b412d7a-f1f7-46ae-b44c-3c6b987a4d91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:25.232617 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b412d7a-f1f7-46ae-b44c-3c6b987a4d91 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:25.293631 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 932/1891] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:52:25 2022] POST /identity/v3/projects => generated 375 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:25.300330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b412d7a-f1f7-46ae-b44c-3c6b987a4d91 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:25.322203 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 960/1892] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:25.327563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06dd7d-ab6c-45ef-a1fd-b8b1520aa689 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:25.376913 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06dd7d-ab6c-45ef-a1fd-b8b1520aa689 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:25.512143 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03396d-9053-4a96-b9c3-dfbd0051e536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:25.513112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03396d-9053-4a96-b9c3-dfbd0051e536 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:25.513490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03396d-9053-4a96-b9c3-dfbd0051e536 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:25.513841 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03396d-9053-4a96-b9c3-dfbd0051e536 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:25.514558 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03396d-9053-4a96-b9c3-dfbd0051e536 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:25.516728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03396d-9053-4a96-b9c3-dfbd0051e536 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:25.529594 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:25.547612 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b412d7a-f1f7-46ae-b44c-3c6b987a4d91 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:25.565516 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9fdbe-5532-4571-916f-744305a784be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:25.571253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9fdbe-5532-4571-916f-744305a784be admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:25.572110 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9fdbe-5532-4571-916f-744305a784be admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:25.572696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9fdbe-5532-4571-916f-744305a784be admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:25.573675 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9fdbe-5532-4571-916f-744305a784be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:25.582077 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9fdbe-5532-4571-916f-744305a784be admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:25.585751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 961/1893] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:52:25 2022] POST /identity/v3/projects => generated 383 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:25.587932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03396d-9053-4a96-b9c3-dfbd0051e536 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:25.592375 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4a03396d-9053-4a96-b9c3-dfbd0051e536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:25.636205 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 933/1894] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:52:25 2022] POST /identity/v3/users => generated 294 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:25.637855 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9fdbe-5532-4571-916f-744305a784be admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:25.654801 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9fdbe-5532-4571-916f-744305a784be admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:25.724195 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a8b06-c445-43a4-a64a-4aa3abaad5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:25.724831 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a8b06-c445-43a4-a64a-4aa3abaad5bd admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:25.725115 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a8b06-c445-43a4-a64a-4aa3abaad5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:25.725510 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a8b06-c445-43a4-a64a-4aa3abaad5bd admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:25.726161 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121a8b06-c445-43a4-a64a-4aa3abaad5bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:25.728167 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121a8b06-c445-43a4-a64a-4aa3abaad5bd admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:25.739880 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03396d-9053-4a96-b9c3-dfbd0051e536 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:25.748086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09ecdd-af79-4a6c-a8a4-d4ae5abb504c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:25.749303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09ecdd-af79-4a6c-a8a4-d4ae5abb504c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:25.749741 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09ecdd-af79-4a6c-a8a4-d4ae5abb504c admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:25.750227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09ecdd-af79-4a6c-a8a4-d4ae5abb504c admin admin] PATH_INFO: `/v3/projects/12fbe05300254a7e83a7d03d1b95ce45` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:25.751969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee09ecdd-af79-4a6c-a8a4-d4ae5abb504c admin admin] RBAC: Authorizing `identity:delete_project(project_id=12fbe05300254a7e83a7d03d1b95ce45)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:25.754937 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee09ecdd-af79-4a6c-a8a4-d4ae5abb504c admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:25.770094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:25.790541 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 934/1895] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:52:25 2022] POST /identity/v3/users => generated 298 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:25.792747 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a8b06-c445-43a4-a64a-4aa3abaad5bd admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:25.833503 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a8b06-c445-43a4-a64a-4aa3abaad5bd admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:25.871422 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-ee09ecdd-af79-4a6c-a8a4-d4ae5abb504c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12fbe05300254a7e83a7d03d1b95ce45', 'request_id': 'req-ee09ecdd-af79-4a6c-a8a4-d4ae5abb504c'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:25.881183 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-ee09ecdd-af79-4a6c-a8a4-d4ae5abb504c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12fbe05300254a7e83a7d03d1b95ce45', 'request_id': 'req-ee09ecdd-af79-4a6c-a8a4-d4ae5abb504c'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:25.906932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-ee09ecdd-af79-4a6c-a8a4-d4ae5abb504c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12fbe05300254a7e83a7d03d1b95ce45', 'request_id': 'req-ee09ecdd-af79-4a6c-a8a4-d4ae5abb504c'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:25.936671 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-ee09ecdd-af79-4a6c-a8a4-d4ae5abb504c admin admin] The token cache is being invalidate because project 12fbe05300254a7e83a7d03d1b95ce45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:52:25.941062 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-ee09ecdd-af79-4a6c-a8a4-d4ae5abb504c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:25.942661 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-ee09ecdd-af79-4a6c-a8a4-d4ae5abb504c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:25.952576 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 962/1896] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:52:25 2022] DELETE /identity/v3/projects/12fbe05300254a7e83a7d03d1b95ce45 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:25.953902 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09ecdd-af79-4a6c-a8a4-d4ae5abb504c admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:25.964048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ee09ecdd-af79-4a6c-a8a4-d4ae5abb504c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:26.023679 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc52afc-6450-4c88-b79f-459a621db3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:26.024315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc52afc-6450-4c88-b79f-459a621db3a3 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:26.024594 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc52afc-6450-4c88-b79f-459a621db3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:26.024865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc52afc-6450-4c88-b79f-459a621db3a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:26.025639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc52afc-6450-4c88-b79f-459a621db3a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:26.027676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc52afc-6450-4c88-b79f-459a621db3a3 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:26.074913 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 935/1897] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:26.076445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a521e60-f49d-487f-9fb8-9f3561649b48 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:26.076564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09ecdd-af79-4a6c-a8a4-d4ae5abb504c admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:26.076731 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a521e60-f49d-487f-9fb8-9f3561649b48 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:26.077056 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a521e60-f49d-487f-9fb8-9f3561649b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:26.105319 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e42c76-4c7b-478b-ba25-28ab743583c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:26.106279 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e42c76-4c7b-478b-ba25-28ab743583c5 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:26.109061 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e42c76-4c7b-478b-ba25-28ab743583c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:26.109515 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e42c76-4c7b-478b-ba25-28ab743583c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:26.110400 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e42c76-4c7b-478b-ba25-28ab743583c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:26.116496 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e42c76-4c7b-478b-ba25-28ab743583c5 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:26.138388 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-4a521e60-f49d-487f-9fb8-9f3561649b48 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:26.140308 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4a521e60-f49d-487f-9fb8-9f3561649b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:26.163538 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 936/1898] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:26.165330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a521e60-f49d-487f-9fb8-9f3561649b48 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:26.170856 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4a521e60-f49d-487f-9fb8-9f3561649b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:26.183153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 963/1899] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:26.186726 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77208ac-24e3-4e55-b3de-91c673b9f5a0 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:26.193539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77208ac-24e3-4e55-b3de-91c673b9f5a0 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:26.194610 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77208ac-24e3-4e55-b3de-91c673b9f5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:26.197725 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a521e60-f49d-487f-9fb8-9f3561649b48 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:26.213794 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a57efe-cb8b-47b9-a2e9-782188288b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:26.214118 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a57efe-cb8b-47b9-a2e9-782188288b8d admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:26.214320 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a57efe-cb8b-47b9-a2e9-782188288b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:26.214576 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a57efe-cb8b-47b9-a2e9-782188288b8d admin admin] PATH_INFO: `/v3/projects/ebdb640062a34b7d9b3323b17420d492/users/dc5801e7e8e7478e9d03165ac63127f5/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:26.228597 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-a77208ac-24e3-4e55-b3de-91c673b9f5a0 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:26.229869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a77208ac-24e3-4e55-b3de-91c673b9f5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:26.233463 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a57efe-cb8b-47b9-a2e9-782188288b8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebdb640062a34b7d9b3323b17420d492, user_id=dc5801e7e8e7478e9d03165ac63127f5, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:26.237490 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a57efe-cb8b-47b9-a2e9-782188288b8d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:26.243078 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 964/1900] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:26.244096 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77208ac-24e3-4e55-b3de-91c673b9f5a0 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:26.258985 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77208ac-24e3-4e55-b3de-91c673b9f5a0 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:26.263398 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 937/1901] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:26 2022] PUT /identity/v3/projects/ebdb640062a34b7d9b3323b17420d492/users/dc5801e7e8e7478e9d03165ac63127f5/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:26.264743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a57efe-cb8b-47b9-a2e9-782188288b8d admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:26.267324 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:26.376386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a57efe-cb8b-47b9-a2e9-782188288b8d admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:26.384747 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22dbcb8-7f8e-42d6-bf4f-c3bde8813754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:26.385865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22dbcb8-7f8e-42d6-bf4f-c3bde8813754 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:26.386253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22dbcb8-7f8e-42d6-bf4f-c3bde8813754 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:26.386807 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22dbcb8-7f8e-42d6-bf4f-c3bde8813754 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:26.387610 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22dbcb8-7f8e-42d6-bf4f-c3bde8813754 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:52:26.388060 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22dbcb8-7f8e-42d6-bf4f-c3bde8813754 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:26.390060 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22dbcb8-7f8e-42d6-bf4f-c3bde8813754 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:26.421947 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 938/1902] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:52:26.421947 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22dbcb8-7f8e-42d6-bf4f-c3bde8813754 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:26.437673 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22dbcb8-7f8e-42d6-bf4f-c3bde8813754 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:26.453421 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0ea96-28ed-4f7d-a1f6-62126f313ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:26.454474 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0ea96-28ed-4f7d-a1f6-62126f313ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:26.455018 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0ea96-28ed-4f7d-a1f6-62126f313ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:26.455528 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0ea96-28ed-4f7d-a1f6-62126f313ba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:26.457256 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d0ea96-28ed-4f7d-a1f6-62126f313ba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:26.459417 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d0ea96-28ed-4f7d-a1f6-62126f313ba6 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:26.477707 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee9655-5518-462c-95cc-83a020e5ad3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:26.478471 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 939/1903] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:26.478877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee9655-5518-462c-95cc-83a020e5ad3a admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:26.478877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee9655-5518-462c-95cc-83a020e5ad3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:26.479183 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee9655-5518-462c-95cc-83a020e5ad3a admin admin] PATH_INFO: `/v3/projects/fdcd8989c9d646cc885aaa9abbce1df9/users/9a5b0fe761dd418687f21a9bf2604338/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:26.479515 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0ea96-28ed-4f7d-a1f6-62126f313ba6 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:26.509645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0ea96-28ed-4f7d-a1f6-62126f313ba6 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:26.514944 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee9655-5518-462c-95cc-83a020e5ad3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdcd8989c9d646cc885aaa9abbce1df9, user_id=9a5b0fe761dd418687f21a9bf2604338, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:26.518193 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e697f-60e0-4ee3-b6de-b41bbea06785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:26.518834 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e697f-60e0-4ee3-b6de-b41bbea06785 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:26.519161 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e697f-60e0-4ee3-b6de-b41bbea06785 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:26.519434 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e697f-60e0-4ee3-b6de-b41bbea06785 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:26.520092 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e697f-60e0-4ee3-b6de-b41bbea06785 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:26.521207 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee9655-5518-462c-95cc-83a020e5ad3a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:26.522008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e697f-60e0-4ee3-b6de-b41bbea06785 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:26.542428 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 965/1904] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:26 2022] PUT /identity/v3/projects/fdcd8989c9d646cc885aaa9abbce1df9/users/9a5b0fe761dd418687f21a9bf2604338/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:26.543632 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee9655-5518-462c-95cc-83a020e5ad3a admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:26.578006 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 940/1905] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:52:26 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:26.579073 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e697f-60e0-4ee3-b6de-b41bbea06785 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:26.610135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:26.669471 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e697f-60e0-4ee3-b6de-b41bbea06785 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:26.671729 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee9655-5518-462c-95cc-83a020e5ad3a admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:26.674753 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3036d5-d30d-4688-94af-5646461d1d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:26.675522 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3036d5-d30d-4688-94af-5646461d1d41 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:26.675807 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3036d5-d30d-4688-94af-5646461d1d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:26.676079 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3036d5-d30d-4688-94af-5646461d1d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:26.676806 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3036d5-d30d-4688-94af-5646461d1d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:26.677747 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c85d9-34b0-447b-8ce7-51c32f6f2abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:26.678438 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3036d5-d30d-4688-94af-5646461d1d41 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:26.678806 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c85d9-34b0-447b-8ce7-51c32f6f2abb admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:26.679253 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c85d9-34b0-447b-8ce7-51c32f6f2abb admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:26.679723 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c85d9-34b0-447b-8ce7-51c32f6f2abb admin admin] PATH_INFO: `/v3/projects/ebdb640062a34b7d9b3323b17420d492/users/dc5801e7e8e7478e9d03165ac63127f5/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:26.682717 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c85d9-34b0-447b-8ce7-51c32f6f2abb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebdb640062a34b7d9b3323b17420d492, user_id=dc5801e7e8e7478e9d03165ac63127f5, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:26.685765 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c85d9-34b0-447b-8ce7-51c32f6f2abb admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:26.690956 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 941/1906] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:26.695129 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3036d5-d30d-4688-94af-5646461d1d41 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:26.701949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 966/1907] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:26 2022] PUT /identity/v3/projects/ebdb640062a34b7d9b3323b17420d492/users/dc5801e7e8e7478e9d03165ac63127f5/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:26.703591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c85d9-34b0-447b-8ce7-51c32f6f2abb admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:26.713629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3036d5-d30d-4688-94af-5646461d1d41 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:26.787145 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aa0218-95e3-469c-b585-8a231342d19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:26.787574 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa0218-95e3-469c-b585-8a231342d19e admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:26.787973 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa0218-95e3-469c-b585-8a231342d19e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:26.788239 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa0218-95e3-469c-b585-8a231342d19e admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:26.789032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aa0218-95e3-469c-b585-8a231342d19e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:26.791426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aa0218-95e3-469c-b585-8a231342d19e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:26.792166 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c85d9-34b0-447b-8ce7-51c32f6f2abb admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:26.801435 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3aae3-e2d2-4eef-ba0a-e81358fa4241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:26.802534 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3aae3-e2d2-4eef-ba0a-e81358fa4241 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:26.803180 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3aae3-e2d2-4eef-ba0a-e81358fa4241 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:26.803684 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3aae3-e2d2-4eef-ba0a-e81358fa4241 admin admin] PATH_INFO: `/v3/projects/fdcd8989c9d646cc885aaa9abbce1df9/users/9a5b0fe761dd418687f21a9bf2604338/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:26.806949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a3aae3-e2d2-4eef-ba0a-e81358fa4241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdcd8989c9d646cc885aaa9abbce1df9, user_id=9a5b0fe761dd418687f21a9bf2604338, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:26.810628 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a3aae3-e2d2-4eef-ba0a-e81358fa4241 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:26.831917 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 942/1908] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:52:26 2022] POST /identity/v3/users => generated 291 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:26.835959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5e841-4f65-4668-be38-634603c468c8 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:26.836503 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5e841-4f65-4668-be38-634603c468c8 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:26.836989 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5e841-4f65-4668-be38-634603c468c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:26.842956 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 967/1909] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:26 2022] PUT /identity/v3/projects/fdcd8989c9d646cc885aaa9abbce1df9/users/9a5b0fe761dd418687f21a9bf2604338/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:26.842956 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3aae3-e2d2-4eef-ba0a-e81358fa4241 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:26.984430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-49c5e841-4f65-4668-be38-634603c468c8 None None] MFA Rules not processed for user `dc5801e7e8e7478e9d03165ac63127f5`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:26.987026 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-49c5e841-4f65-4668-be38-634603c468c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:26.992960 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:27.114984 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3aae3-e2d2-4eef-ba0a-e81358fa4241 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:27.121505 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dabb18-0dbf-437b-988b-e7785d2d3473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:27.123349 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dabb18-0dbf-437b-988b-e7785d2d3473 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:27.124045 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dabb18-0dbf-437b-988b-e7785d2d3473 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:27.124717 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dabb18-0dbf-437b-988b-e7785d2d3473 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:27.126251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dabb18-0dbf-437b-988b-e7785d2d3473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:27.129644 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dabb18-0dbf-437b-988b-e7785d2d3473 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:27.150324 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 968/1910] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:27.152515 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18794eac-f0ab-41c3-b9c7-142ef22962f5 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:27.152994 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18794eac-f0ab-41c3-b9c7-142ef22962f5 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:27.153924 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18794eac-f0ab-41c3-b9c7-142ef22962f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:27.187207 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 943/1911] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:27.189043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5e841-4f65-4668-be38-634603c468c8 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:27.215454 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5e841-4f65-4668-be38-634603c468c8 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:27.219742 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-18794eac-f0ab-41c3-b9c7-142ef22962f5 None None] MFA Rules not processed for user `9a5b0fe761dd418687f21a9bf2604338`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:27.221658 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-18794eac-f0ab-41c3-b9c7-142ef22962f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:27.225392 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa99204-7bb0-41ad-ad25-b4d2ce6576e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:27.226171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa99204-7bb0-41ad-ad25-b4d2ce6576e0 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:27.229481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa99204-7bb0-41ad-ad25-b4d2ce6576e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:27.229899 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa99204-7bb0-41ad-ad25-b4d2ce6576e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:27.230695 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa99204-7bb0-41ad-ad25-b4d2ce6576e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:27.243035 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa99204-7bb0-41ad-ad25-b4d2ce6576e0 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:27.268116 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 944/1912] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:27.269336 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa99204-7bb0-41ad-ad25-b4d2ce6576e0 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:27.271885 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-2fa99204-7bb0-41ad-ad25-b4d2ce6576e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:27.343846 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:27.413470 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 969/1913] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:27 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:27.413470 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18794eac-f0ab-41c3-b9c7-142ef22962f5 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:27.446771 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa99204-7bb0-41ad-ad25-b4d2ce6576e0 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:27.453629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b82e65-cc65-47c0-9794-d87c28a3c9c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:27.454555 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18794eac-f0ab-41c3-b9c7-142ef22962f5 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:27.455121 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b82e65-cc65-47c0-9794-d87c28a3c9c5 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:27.455745 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b82e65-cc65-47c0-9794-d87c28a3c9c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:27.456376 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b82e65-cc65-47c0-9794-d87c28a3c9c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:27.460097 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-23b82e65-cc65-47c0-9794-d87c28a3c9c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:27.462094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07d390f-b484-4504-9b7c-5b9a82f0915d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:27.463068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d390f-b484-4504-9b7c-5b9a82f0915d admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:27.463486 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d390f-b484-4504-9b7c-5b9a82f0915d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:27.463921 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07d390f-b484-4504-9b7c-5b9a82f0915d admin admin] PATH_INFO: `/v3/projects/f9f5c6bf70c947a8ab853038b57e3d60/users/2cc6b2457444456096012d2ebce81260/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:27.485045 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07d390f-b484-4504-9b7c-5b9a82f0915d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9f5c6bf70c947a8ab853038b57e3d60, user_id=2cc6b2457444456096012d2ebce81260, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:27.489610 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07d390f-b484-4504-9b7c-5b9a82f0915d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:27.510639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 970/1914] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:27 2022] PUT /identity/v3/projects/f9f5c6bf70c947a8ab853038b57e3d60/users/2cc6b2457444456096012d2ebce81260/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:27.512330 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925ae4b-8d8a-4f6e-aef8-c15304a36ed0 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:27.512790 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925ae4b-8d8a-4f6e-aef8-c15304a36ed0 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:27.513255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925ae4b-8d8a-4f6e-aef8-c15304a36ed0 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:27.515742 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 971/1915] 10.222.0.71 () {54 vars in 921 bytes} [Fri Aug 26 08:52:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:52:27.519851 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925ae4b-8d8a-4f6e-aef8-c15304a36ed0 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:27.524673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-0925ae4b-8d8a-4f6e-aef8-c15304a36ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:27.621936 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925ae4b-8d8a-4f6e-aef8-c15304a36ed0 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:27.626530 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b82e65-cc65-47c0-9794-d87c28a3c9c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:27.627000 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5c9f4-72ed-445d-bb6b-9d824c4dbb69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:27.627830 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5c9f4-72ed-445d-bb6b-9d824c4dbb69 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:27.628114 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5c9f4-72ed-445d-bb6b-9d824c4dbb69 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:27.628537 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5c9f4-72ed-445d-bb6b-9d824c4dbb69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:27.631000 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b82e65-cc65-47c0-9794-d87c28a3c9c5 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:27.659232 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 945/1916] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:27.661986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c1e3d-20e8-4b2b-b6bb-c1caca401065 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:27.662613 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c1e3d-20e8-4b2b-b6bb-c1caca401065 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:27.663482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c1e3d-20e8-4b2b-b6bb-c1caca401065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:27.696532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-446c1e3d-20e8-4b2b-b6bb-c1caca401065 None None] MFA Rules not processed for user `f16d1768715a41648af3fe6afadd7477`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:27.699049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-446c1e3d-20e8-4b2b-b6bb-c1caca401065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:27.721841 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:27.819588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f5c9f4-72ed-445d-bb6b-9d824c4dbb69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:27.822939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f5c9f4-72ed-445d-bb6b-9d824c4dbb69 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:27.861655 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 972/1917] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:27 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:27.864210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9ba93-5398-4e29-88c6-1b1151daae59 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:27.864609 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9ba93-5398-4e29-88c6-1b1151daae59 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:27.866552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9ba93-5398-4e29-88c6-1b1151daae59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:27.902773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 946/1918] 10.222.0.71 () {58 vars in 1018 bytes} [Fri Aug 26 08:52:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:27.946843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-c8e9ba93-5398-4e29-88c6-1b1151daae59 None None] MFA Rules not processed for user `2cc6b2457444456096012d2ebce81260`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:27.950582 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c8e9ba93-5398-4e29-88c6-1b1151daae59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:28.165397 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c1e3d-20e8-4b2b-b6bb-c1caca401065 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:28.167632 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 973/1919] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:28.315132 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9ba93-5398-4e29-88c6-1b1151daae59 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:28.344590 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9ba93-5398-4e29-88c6-1b1151daae59 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:28.349069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5daeee-3a65-47fd-888a-50c737e5f1ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:28.349730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5daeee-3a65-47fd-888a-50c737e5f1ce service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:28.350105 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5daeee-3a65-47fd-888a-50c737e5f1ce service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:28.350220 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c1e3d-20e8-4b2b-b6bb-c1caca401065 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:28.350303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5daeee-3a65-47fd-888a-50c737e5f1ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:28.354079 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-af5daeee-3a65-47fd-888a-50c737e5f1ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:28.371668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d299f-cba3-4a7a-8e52-75735444d383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:28.372819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d299f-cba3-4a7a-8e52-75735444d383 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:28.373187 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d299f-cba3-4a7a-8e52-75735444d383 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:28.375473 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d299f-cba3-4a7a-8e52-75735444d383 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:28.377562 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86d299f-cba3-4a7a-8e52-75735444d383 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:28.379939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86d299f-cba3-4a7a-8e52-75735444d383 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:28.444034 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:28.471349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 947/1920] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:52:28 2022] POST /identity/v3/projects => generated 381 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:28.472249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d299f-cba3-4a7a-8e52-75735444d383 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:28.476771 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b86d299f-cba3-4a7a-8e52-75735444d383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:28.572067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5daeee-3a65-47fd-888a-50c737e5f1ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:28.574033 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5daeee-3a65-47fd-888a-50c737e5f1ce service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:28.596187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 974/1921] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:28 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:28.597508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5daeee-3a65-47fd-888a-50c737e5f1ce service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:28.618171 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5daeee-3a65-47fd-888a-50c737e5f1ce service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:28.691948 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d299f-cba3-4a7a-8e52-75735444d383 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:28.715767 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c1feb-2570-42c0-9b5e-a9a7ba0beb8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:28.717540 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c1feb-2570-42c0-9b5e-a9a7ba0beb8a service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:28.719077 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c1feb-2570-42c0-9b5e-a9a7ba0beb8a service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:28.719945 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c1feb-2570-42c0-9b5e-a9a7ba0beb8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:28.745659 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c1feb-2570-42c0-9b5e-a9a7ba0beb8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:28.749860 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c1feb-2570-42c0-9b5e-a9a7ba0beb8a service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:28.766580 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:28.789914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 948/1922] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:28.792364 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f2021-fad6-4194-83cf-7baae068e091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:28.793217 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f2021-fad6-4194-83cf-7baae068e091 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:28.794345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f2021-fad6-4194-83cf-7baae068e091 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:28.795330 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f2021-fad6-4194-83cf-7baae068e091 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:28.796816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f2021-fad6-4194-83cf-7baae068e091 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:28.805086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f2021-fad6-4194-83cf-7baae068e091 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:28.860531 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 975/1923] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:52:28 2022] POST /identity/v3/users => generated 297 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:28.863347 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c1feb-2570-42c0-9b5e-a9a7ba0beb8a service nova] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:28.880064 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c1feb-2570-42c0-9b5e-a9a7ba0beb8a service nova] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:28.893672 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442dd11d-3e39-4905-8908-94b01b31e55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:28.894534 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442dd11d-3e39-4905-8908-94b01b31e55c admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:28.897083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442dd11d-3e39-4905-8908-94b01b31e55c admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:28.898868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442dd11d-3e39-4905-8908-94b01b31e55c admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:28.901564 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442dd11d-3e39-4905-8908-94b01b31e55c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:28.906755 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442dd11d-3e39-4905-8908-94b01b31e55c admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:28.968522 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 949/1924] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:28.977860 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f2021-fad6-4194-83cf-7baae068e091 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:29.023196 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f2021-fad6-4194-83cf-7baae068e091 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:29.045990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4cebc9-e44f-4f64-948a-2d6d13590fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:29.047534 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cebc9-e44f-4f64-948a-2d6d13590fc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:29.048041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cebc9-e44f-4f64-948a-2d6d13590fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:29.048575 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cebc9-e44f-4f64-948a-2d6d13590fc4 admin admin] PATH_INFO: `/v3/projects/2c6ba6c6a3c242d9936865a71c167c1d/users/d527662045b5483aa39c35c51bf6855a/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:29.093722 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4cebc9-e44f-4f64-948a-2d6d13590fc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c6ba6c6a3c242d9936865a71c167c1d, user_id=d527662045b5483aa39c35c51bf6855a, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:29.099071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4cebc9-e44f-4f64-948a-2d6d13590fc4 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:29.132178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 976/1925] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:28 2022] PUT /identity/v3/projects/2c6ba6c6a3c242d9936865a71c167c1d/users/d527662045b5483aa39c35c51bf6855a/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:29.140024 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442dd11d-3e39-4905-8908-94b01b31e55c admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:29.163154 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442dd11d-3e39-4905-8908-94b01b31e55c admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:29.248280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4cebc9-e44f-4f64-948a-2d6d13590fc4 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:29.333475 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f1555-e299-43fe-bfeb-0c257695b377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:29.334122 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f1555-e299-43fe-bfeb-0c257695b377 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:29.334405 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f1555-e299-43fe-bfeb-0c257695b377 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:29.334718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f1555-e299-43fe-bfeb-0c257695b377 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:29.339075 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010f1555-e299-43fe-bfeb-0c257695b377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:29.347499 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010f1555-e299-43fe-bfeb-0c257695b377 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:29.386728 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 950/1926] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:29 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:29.388950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f1555-e299-43fe-bfeb-0c257695b377 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:29.405690 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f1555-e299-43fe-bfeb-0c257695b377 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:29.414819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850258dd-4a9f-4fac-839b-28665ac9b9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:29.416288 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850258dd-4a9f-4fac-839b-28665ac9b9aa admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:29.417245 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4cebc9-e44f-4f64-948a-2d6d13590fc4 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:29.419662 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850258dd-4a9f-4fac-839b-28665ac9b9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:29.421317 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850258dd-4a9f-4fac-839b-28665ac9b9aa admin admin] PATH_INFO: `/v3/projects/2c6ba6c6a3c242d9936865a71c167c1d/users/d527662045b5483aa39c35c51bf6855a/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:29.425087 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a41cc8-7190-41c7-abbe-5edba9987f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:29.425383 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850258dd-4a9f-4fac-839b-28665ac9b9aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c6ba6c6a3c242d9936865a71c167c1d, user_id=d527662045b5483aa39c35c51bf6855a, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:29.425559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a41cc8-7190-41c7-abbe-5edba9987f79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:29.425559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a41cc8-7190-41c7-abbe-5edba9987f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:29.425559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a41cc8-7190-41c7-abbe-5edba9987f79 admin admin] PATH_INFO: `/v3/users/cb1ccbd55efc456eb49604003fcabb8b` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:29.427088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a41cc8-7190-41c7-abbe-5edba9987f79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb1ccbd55efc456eb49604003fcabb8b)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:29.429234 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a41cc8-7190-41c7-abbe-5edba9987f79 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:29.432800 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850258dd-4a9f-4fac-839b-28665ac9b9aa admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:29.519921 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 951/1927] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:29 2022] PUT /identity/v3/projects/2c6ba6c6a3c242d9936865a71c167c1d/users/d527662045b5483aa39c35c51bf6855a/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:29.538067 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f76f0-0378-4fd7-b610-ca7f4bebc440 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:29.542914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f76f0-0378-4fd7-b610-ca7f4bebc440 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:29.542914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f76f0-0378-4fd7-b610-ca7f4bebc440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:29.592086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-14a41cc8-7190-41c7-abbe-5edba9987f79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb1ccbd55efc456eb49604003fcabb8b'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:29.600890 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b08f76f0-0378-4fd7-b610-ca7f4bebc440 None None] MFA Rules not processed for user `d527662045b5483aa39c35c51bf6855a`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:29.602879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b08f76f0-0378-4fd7-b610-ca7f4bebc440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:29.641382 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-14a41cc8-7190-41c7-abbe-5edba9987f79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb1ccbd55efc456eb49604003fcabb8b'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:29.659256 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-14a41cc8-7190-41c7-abbe-5edba9987f79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb1ccbd55efc456eb49604003fcabb8b'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:29.674048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-14a41cc8-7190-41c7-abbe-5edba9987f79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb1ccbd55efc456eb49604003fcabb8b'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:29.675478 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-14a41cc8-7190-41c7-abbe-5edba9987f79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb1ccbd55efc456eb49604003fcabb8b'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:29.694231 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 977/1928] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:52:29 2022] DELETE /identity/v3/users/cb1ccbd55efc456eb49604003fcabb8b => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:29.701419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f63596-7bae-41a4-88cc-d413f93bbcac None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:29.711209 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f63596-7bae-41a4-88cc-d413f93bbcac None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:29.711209 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f63596-7bae-41a4-88cc-d413f93bbcac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:29.759080 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-b2f63596-7bae-41a4-88cc-d413f93bbcac None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:29.760717 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b2f63596-7bae-41a4-88cc-d413f93bbcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:29.770471 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:29.833924 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:29.835369 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 952/1929] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:29.848937 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f76f0-0378-4fd7-b610-ca7f4bebc440 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:29.850489 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b08f76f0-0378-4fd7-b610-ca7f4bebc440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:29.917257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 978/1930] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:29.918627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f63596-7bae-41a4-88cc-d413f93bbcac None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:29.920874 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b2f63596-7bae-41a4-88cc-d413f93bbcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:29.967371 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f76f0-0378-4fd7-b610-ca7f4bebc440 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:30.019617 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f63596-7bae-41a4-88cc-d413f93bbcac None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:30.031337 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8106c-a9a8-45f0-bf15-eea58e0bdf80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:30.032077 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8106c-a9a8-45f0-bf15-eea58e0bdf80 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:30.032562 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8106c-a9a8-45f0-bf15-eea58e0bdf80 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:30.032849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8106c-a9a8-45f0-bf15-eea58e0bdf80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:30.037392 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-42c8106c-a9a8-45f0-bf15-eea58e0bdf80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:30.073995 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce15e4-6dee-4e06-8cf1-61c05e5aeedb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:30.075146 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce15e4-6dee-4e06-8cf1-61c05e5aeedb service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:30.075729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce15e4-6dee-4e06-8cf1-61c05e5aeedb service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:30.076541 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce15e4-6dee-4e06-8cf1-61c05e5aeedb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:30.160821 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c8106c-a9a8-45f0-bf15-eea58e0bdf80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:30.162025 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c8106c-a9a8-45f0-bf15-eea58e0bdf80 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:30.172764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 979/1931] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:30.173842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8106c-a9a8-45f0-bf15-eea58e0bdf80 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:30.175765 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-42c8106c-a9a8-45f0-bf15-eea58e0bdf80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:30.215413 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ce15e4-6dee-4e06-8cf1-61c05e5aeedb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:30.218490 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ce15e4-6dee-4e06-8cf1-61c05e5aeedb service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:30.260886 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 953/1932] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:52:30.292636 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8106c-a9a8-45f0-bf15-eea58e0bdf80 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:30.298640 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90560cf4-4c7e-4a8f-904e-94a382412f1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:30.299548 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90560cf4-4c7e-4a8f-904e-94a382412f1a service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:30.300018 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90560cf4-4c7e-4a8f-904e-94a382412f1a service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:30.300572 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90560cf4-4c7e-4a8f-904e-94a382412f1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:30.319213 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90560cf4-4c7e-4a8f-904e-94a382412f1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:30.319213 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90560cf4-4c7e-4a8f-904e-94a382412f1a service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:30.332807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 980/1933] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:31.084505 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce15e4-6dee-4e06-8cf1-61c05e5aeedb service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:31.086537 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-27ce15e4-6dee-4e06-8cf1-61c05e5aeedb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:31.110163 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce15e4-6dee-4e06-8cf1-61c05e5aeedb service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:31.117347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90560cf4-4c7e-4a8f-904e-94a382412f1a service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:31.120220 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-90560cf4-4c7e-4a8f-904e-94a382412f1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:31.121722 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d047f7-2c42-420f-bc62-a5cf0127bbb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:31.122413 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d047f7-2c42-420f-bc62-a5cf0127bbb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:31.122741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d047f7-2c42-420f-bc62-a5cf0127bbb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:31.123071 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d047f7-2c42-420f-bc62-a5cf0127bbb3 admin admin] PATH_INFO: `/v3/users/2cc6b2457444456096012d2ebce81260` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:31.124435 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d047f7-2c42-420f-bc62-a5cf0127bbb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cc6b2457444456096012d2ebce81260)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:31.126629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d047f7-2c42-420f-bc62-a5cf0127bbb3 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:31.146854 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90560cf4-4c7e-4a8f-904e-94a382412f1a service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:31.163432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b992ac7-f9af-4496-97b3-6e3a33fa920d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:31.164398 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b992ac7-f9af-4496-97b3-6e3a33fa920d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:31.164713 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b992ac7-f9af-4496-97b3-6e3a33fa920d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:31.165003 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b992ac7-f9af-4496-97b3-6e3a33fa920d admin admin] PATH_INFO: `/v3/projects/381a7b4f9d6f406781c9f3184f9c2491` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:31.166678 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b992ac7-f9af-4496-97b3-6e3a33fa920d admin admin] RBAC: Authorizing `identity:delete_project(project_id=381a7b4f9d6f406781c9f3184f9c2491)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:31.168822 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b992ac7-f9af-4496-97b3-6e3a33fa920d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:31.181827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:31.189747 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-76d047f7-2c42-420f-bc62-a5cf0127bbb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cc6b2457444456096012d2ebce81260'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:31.204086 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-76d047f7-2c42-420f-bc62-a5cf0127bbb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cc6b2457444456096012d2ebce81260'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:31.213403 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-76d047f7-2c42-420f-bc62-a5cf0127bbb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cc6b2457444456096012d2ebce81260'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:31.214251 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-76d047f7-2c42-420f-bc62-a5cf0127bbb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cc6b2457444456096012d2ebce81260'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:31.215013 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-76d047f7-2c42-420f-bc62-a5cf0127bbb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cc6b2457444456096012d2ebce81260'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:31.218253 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:31.239137 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0b992ac7-f9af-4496-97b3-6e3a33fa920d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '381a7b4f9d6f406781c9f3184f9c2491', 'request_id': 'req-0b992ac7-f9af-4496-97b3-6e3a33fa920d'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:31.239785 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 954/1934] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:52:31 2022] DELETE /identity/v3/users/2cc6b2457444456096012d2ebce81260 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:31.244859 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368b747-170f-4e7b-aa29-f55163eff68f None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:31.245144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368b747-170f-4e7b-aa29-f55163eff68f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:31.245420 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368b747-170f-4e7b-aa29-f55163eff68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:31.247021 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0b992ac7-f9af-4496-97b3-6e3a33fa920d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '381a7b4f9d6f406781c9f3184f9c2491', 'request_id': 'req-0b992ac7-f9af-4496-97b3-6e3a33fa920d'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:31.258823 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0b992ac7-f9af-4496-97b3-6e3a33fa920d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '381a7b4f9d6f406781c9f3184f9c2491', 'request_id': 'req-0b992ac7-f9af-4496-97b3-6e3a33fa920d'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:31.276623 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-1368b747-170f-4e7b-aa29-f55163eff68f None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:31.277776 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1368b747-170f-4e7b-aa29-f55163eff68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:31.278590 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0b992ac7-f9af-4496-97b3-6e3a33fa920d admin admin] The token cache is being invalidate because project 381a7b4f9d6f406781c9f3184f9c2491 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:52:31.279064 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0b992ac7-f9af-4496-97b3-6e3a33fa920d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:31.280195 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0b992ac7-f9af-4496-97b3-6e3a33fa920d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:31.282460 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 981/1935] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:52:31 2022] DELETE /identity/v3/projects/381a7b4f9d6f406781c9f3184f9c2491 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:52:31.288577 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b992ac7-f9af-4496-97b3-6e3a33fa920d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:31.290755 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-0b992ac7-f9af-4496-97b3-6e3a33fa920d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:31.396291 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 955/1936] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:31.405324 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1368b747-170f-4e7b-aa29-f55163eff68f None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:31.407562 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1368b747-170f-4e7b-aa29-f55163eff68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:31.409753 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b992ac7-f9af-4496-97b3-6e3a33fa920d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:31.415536 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa0098-1daa-4bc5-a027-9a50ca633697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:31.416377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa0098-1daa-4bc5-a027-9a50ca633697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:31.416724 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa0098-1daa-4bc5-a027-9a50ca633697 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:31.417078 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa0098-1daa-4bc5-a027-9a50ca633697 admin admin] PATH_INFO: `/v3/users/9a5b0fe761dd418687f21a9bf2604338` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:31.418526 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fa0098-1daa-4bc5-a027-9a50ca633697 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a5b0fe761dd418687f21a9bf2604338)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:31.420715 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fa0098-1daa-4bc5-a027-9a50ca633697 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:31.547758 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1368b747-170f-4e7b-aa29-f55163eff68f None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:31.551941 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-90fa0098-1daa-4bc5-a027-9a50ca633697 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a5b0fe761dd418687f21a9bf2604338'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:31.557025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32fc3f-7ad8-411d-a8a7-efd251a86a3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:31.558017 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32fc3f-7ad8-411d-a8a7-efd251a86a3a service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:31.558386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32fc3f-7ad8-411d-a8a7-efd251a86a3a service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:31.558758 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32fc3f-7ad8-411d-a8a7-efd251a86a3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:31.579365 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-90fa0098-1daa-4bc5-a027-9a50ca633697 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a5b0fe761dd418687f21a9bf2604338'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:31.584502 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32fc3f-7ad8-411d-a8a7-efd251a86a3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:31.586902 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32fc3f-7ad8-411d-a8a7-efd251a86a3a service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:31.598359 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-90fa0098-1daa-4bc5-a027-9a50ca633697 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a5b0fe761dd418687f21a9bf2604338'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:31.603331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 956/1937] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:31.609017 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-90fa0098-1daa-4bc5-a027-9a50ca633697 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a5b0fe761dd418687f21a9bf2604338'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:31.609968 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-90fa0098-1daa-4bc5-a027-9a50ca633697 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a5b0fe761dd418687f21a9bf2604338'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:31.618239 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 982/1938] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:52:31 2022] DELETE /identity/v3/users/9a5b0fe761dd418687f21a9bf2604338 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:32.009479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32fc3f-7ad8-411d-a8a7-efd251a86a3a service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:32.015657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-fb32fc3f-7ad8-411d-a8a7-efd251a86a3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:32.107168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa0098-1daa-4bc5-a027-9a50ca633697 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:32.109591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-90fa0098-1daa-4bc5-a027-9a50ca633697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:32.154570 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32fc3f-7ad8-411d-a8a7-efd251a86a3a service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:32.184581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:32.199815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:32.261472 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa0098-1daa-4bc5-a027-9a50ca633697 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:32.269950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bce78-25ba-4901-a897-8741ff25402e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:32.270863 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bce78-25ba-4901-a897-8741ff25402e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:32.271398 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bce78-25ba-4901-a897-8741ff25402e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:32.272153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bce78-25ba-4901-a897-8741ff25402e admin admin] PATH_INFO: `/v3/projects/fdcd8989c9d646cc885aaa9abbce1df9` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:32.274031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8bce78-25ba-4901-a897-8741ff25402e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdcd8989c9d646cc885aaa9abbce1df9)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:32.276492 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8bce78-25ba-4901-a897-8741ff25402e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:32.291073 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea5b1aa-50b0-40ce-88c5-d350a5c6e875 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:32.291741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea5b1aa-50b0-40ce-88c5-d350a5c6e875 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:32.292193 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea5b1aa-50b0-40ce-88c5-d350a5c6e875 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:32.292367 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea5b1aa-50b0-40ce-88c5-d350a5c6e875 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:32.297761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8ea5b1aa-50b0-40ce-88c5-d350a5c6e875 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:32.359816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0c8bce78-25ba-4901-a897-8741ff25402e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdcd8989c9d646cc885aaa9abbce1df9', 'request_id': 'req-0c8bce78-25ba-4901-a897-8741ff25402e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:32.372431 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0c8bce78-25ba-4901-a897-8741ff25402e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdcd8989c9d646cc885aaa9abbce1df9', 'request_id': 'req-0c8bce78-25ba-4901-a897-8741ff25402e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:32.395054 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0c8bce78-25ba-4901-a897-8741ff25402e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdcd8989c9d646cc885aaa9abbce1df9', 'request_id': 'req-0c8bce78-25ba-4901-a897-8741ff25402e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:32.432394 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0c8bce78-25ba-4901-a897-8741ff25402e admin admin] The token cache is being invalidate because project fdcd8989c9d646cc885aaa9abbce1df9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:52:32.432926 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0c8bce78-25ba-4901-a897-8741ff25402e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:32.434132 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0c8bce78-25ba-4901-a897-8741ff25402e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:32.437800 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 983/1939] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:52:32 2022] DELETE /identity/v3/projects/fdcd8989c9d646cc885aaa9abbce1df9 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:32.439009 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bce78-25ba-4901-a897-8741ff25402e admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:32.440793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-0c8bce78-25ba-4901-a897-8741ff25402e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:32.456142 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea5b1aa-50b0-40ce-88c5-d350a5c6e875 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:32.458376 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea5b1aa-50b0-40ce-88c5-d350a5c6e875 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:32.472375 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 957/1940] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:32 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:32.473898 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a88f39-5813-41d2-a570-8b03e4950357 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:32.474340 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a88f39-5813-41d2-a570-8b03e4950357 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:32.474675 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a88f39-5813-41d2-a570-8b03e4950357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:32.512111 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-45a88f39-5813-41d2-a570-8b03e4950357 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:32.513479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-45a88f39-5813-41d2-a570-8b03e4950357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:32.580493 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bce78-25ba-4901-a897-8741ff25402e admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:32.596036 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437416e-cc04-4fcf-a555-0a5ee734bb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:32.598361 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437416e-cc04-4fcf-a555-0a5ee734bb62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:32.600095 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437416e-cc04-4fcf-a555-0a5ee734bb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:32.601632 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437416e-cc04-4fcf-a555-0a5ee734bb62 admin admin] PATH_INFO: `/v3/projects/f9f5c6bf70c947a8ab853038b57e3d60` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:32.603770 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7437416e-cc04-4fcf-a555-0a5ee734bb62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9f5c6bf70c947a8ab853038b57e3d60)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:32.609289 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7437416e-cc04-4fcf-a555-0a5ee734bb62 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:32.664454 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 958/1941] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:32.674815 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508a007-64da-45bf-ae65-d37ba7641f63 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:32.675374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508a007-64da-45bf-ae65-d37ba7641f63 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:32.675843 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508a007-64da-45bf-ae65-d37ba7641f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:32.724388 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-f508a007-64da-45bf-ae65-d37ba7641f63 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:32.726358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f508a007-64da-45bf-ae65-d37ba7641f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:32.777441 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7437416e-cc04-4fcf-a555-0a5ee734bb62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9f5c6bf70c947a8ab853038b57e3d60', 'request_id': 'req-7437416e-cc04-4fcf-a555-0a5ee734bb62'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:32.793203 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7437416e-cc04-4fcf-a555-0a5ee734bb62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9f5c6bf70c947a8ab853038b57e3d60', 'request_id': 'req-7437416e-cc04-4fcf-a555-0a5ee734bb62'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:32.808993 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7437416e-cc04-4fcf-a555-0a5ee734bb62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9f5c6bf70c947a8ab853038b57e3d60', 'request_id': 'req-7437416e-cc04-4fcf-a555-0a5ee734bb62'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:32.836300 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:32.842150 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7437416e-cc04-4fcf-a555-0a5ee734bb62 admin admin] The token cache is being invalidate because project f9f5c6bf70c947a8ab853038b57e3d60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:52:32.843106 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7437416e-cc04-4fcf-a555-0a5ee734bb62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:32.844517 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7437416e-cc04-4fcf-a555-0a5ee734bb62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:32.846409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 984/1942] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:52:32 2022] DELETE /identity/v3/projects/f9f5c6bf70c947a8ab853038b57e3d60 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:32.852756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437416e-cc04-4fcf-a555-0a5ee734bb62 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:32.854417 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-7437416e-cc04-4fcf-a555-0a5ee734bb62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:32.860552 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 959/1943] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:52:32.868111 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508a007-64da-45bf-ae65-d37ba7641f63 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:32.873922 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437416e-cc04-4fcf-a555-0a5ee734bb62 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:32.880107 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9571bd-ab83-4d58-a3f3-243b5749b78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:32.880442 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508a007-64da-45bf-ae65-d37ba7641f63 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:32.880884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9571bd-ab83-4d58-a3f3-243b5749b78f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:32.881267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9571bd-ab83-4d58-a3f3-243b5749b78f admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:32.881557 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9571bd-ab83-4d58-a3f3-243b5749b78f admin admin] PATH_INFO: `/v3/users/d527662045b5483aa39c35c51bf6855a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:32.883246 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9571bd-ab83-4d58-a3f3-243b5749b78f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d527662045b5483aa39c35c51bf6855a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:32.885612 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2620994e-76a8-465d-aa14-e64aec91e476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:32.886140 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9571bd-ab83-4d58-a3f3-243b5749b78f admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:32.886674 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620994e-76a8-465d-aa14-e64aec91e476 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:32.887054 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620994e-76a8-465d-aa14-e64aec91e476 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:32.887391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620994e-76a8-465d-aa14-e64aec91e476 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:32.888138 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2620994e-76a8-465d-aa14-e64aec91e476 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:52:32.888502 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2620994e-76a8-465d-aa14-e64aec91e476 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:32.890725 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2620994e-76a8-465d-aa14-e64aec91e476 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:32.893303 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:32.918743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 960/1944] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:52:32.926234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2620994e-76a8-465d-aa14-e64aec91e476 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:32.983733 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2620994e-76a8-465d-aa14-e64aec91e476 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:33.005059 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5b9571bd-ab83-4d58-a3f3-243b5749b78f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd527662045b5483aa39c35c51bf6855a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:33.005698 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b19b8-7dda-4a84-aa76-ff91332d766f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:33.006712 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b19b8-7dda-4a84-aa76-ff91332d766f admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:33.008433 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b19b8-7dda-4a84-aa76-ff91332d766f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:33.008433 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b19b8-7dda-4a84-aa76-ff91332d766f admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:33.009070 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b19b8-7dda-4a84-aa76-ff91332d766f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:33.014202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b19b8-7dda-4a84-aa76-ff91332d766f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:33.037962 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5b9571bd-ab83-4d58-a3f3-243b5749b78f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd527662045b5483aa39c35c51bf6855a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:33.069556 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5b9571bd-ab83-4d58-a3f3-243b5749b78f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd527662045b5483aa39c35c51bf6855a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:33.108889 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5b9571bd-ab83-4d58-a3f3-243b5749b78f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd527662045b5483aa39c35c51bf6855a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:33.119518 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5b9571bd-ab83-4d58-a3f3-243b5749b78f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd527662045b5483aa39c35c51bf6855a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:33.123977 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 961/1945] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:52:32 2022] POST /identity/v3/projects => generated 381 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:52:33.129725 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b19b8-7dda-4a84-aa76-ff91332d766f admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:33.133573 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 985/1946] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:52:32 2022] DELETE /identity/v3/users/d527662045b5483aa39c35c51bf6855a => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:52:33.135888 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4e9b19b8-7dda-4a84-aa76-ff91332d766f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:33.233391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b19b8-7dda-4a84-aa76-ff91332d766f admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:33.492075 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f96c5-ee26-4ccb-97ca-027f829d383b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:33.492655 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f96c5-ee26-4ccb-97ca-027f829d383b admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:33.492940 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f96c5-ee26-4ccb-97ca-027f829d383b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:33.493218 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f96c5-ee26-4ccb-97ca-027f829d383b admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:33.493910 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f96c5-ee26-4ccb-97ca-027f829d383b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:33.496415 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f96c5-ee26-4ccb-97ca-027f829d383b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:33.543871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 962/1947] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:52:33 2022] POST /identity/v3/users => generated 291 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:52:33.549090 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9571bd-ab83-4d58-a3f3-243b5749b78f admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:33.571660 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9571bd-ab83-4d58-a3f3-243b5749b78f admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:33.588070 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561bab9-7cda-4239-b938-f353fa0d36f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:33.588794 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561bab9-7cda-4239-b938-f353fa0d36f2 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:33.589615 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561bab9-7cda-4239-b938-f353fa0d36f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:33.589903 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561bab9-7cda-4239-b938-f353fa0d36f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:33.590603 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4561bab9-7cda-4239-b938-f353fa0d36f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:33.592491 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4561bab9-7cda-4239-b938-f353fa0d36f2 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:33.628646 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 986/1948] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:33 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 08:52:33.630662 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f96c5-ee26-4ccb-97ca-027f829d383b admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:33.651187 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f96c5-ee26-4ccb-97ca-027f829d383b admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:33.659559 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35880030-3216-4a86-8328-78f7cd6ea9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:33.660325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35880030-3216-4a86-8328-78f7cd6ea9c4 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:33.660779 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35880030-3216-4a86-8328-78f7cd6ea9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:33.661060 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35880030-3216-4a86-8328-78f7cd6ea9c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:33.661667 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35880030-3216-4a86-8328-78f7cd6ea9c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:33.665228 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35880030-3216-4a86-8328-78f7cd6ea9c4 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:33.695364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 963/1949] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:33 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:33.716382 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561bab9-7cda-4239-b938-f353fa0d36f2 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:33.731402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561bab9-7cda-4239-b938-f353fa0d36f2 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:33.741097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316df66-dd96-4e62-aab7-cc509002a242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:33.741930 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316df66-dd96-4e62-aab7-cc509002a242 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:33.745759 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316df66-dd96-4e62-aab7-cc509002a242 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:33.746131 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316df66-dd96-4e62-aab7-cc509002a242 admin admin] PATH_INFO: `/v3/projects/cb00ffae95b94eb693c653407294a097/users/fc457fd883664398bdfe033e8fd8c02b/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:33.779957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e316df66-dd96-4e62-aab7-cc509002a242 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb00ffae95b94eb693c653407294a097, user_id=fc457fd883664398bdfe033e8fd8c02b, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:33.783369 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e316df66-dd96-4e62-aab7-cc509002a242 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:33.820216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 987/1950] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:33 2022] PUT /identity/v3/projects/cb00ffae95b94eb693c653407294a097/users/fc457fd883664398bdfe033e8fd8c02b/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:33.840854 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e556e-1505-49c8-92fd-ff34b308f51e None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:33.841111 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e556e-1505-49c8-92fd-ff34b308f51e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:33.841397 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e556e-1505-49c8-92fd-ff34b308f51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:33.915339 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-9d7e556e-1505-49c8-92fd-ff34b308f51e None None] MFA Rules not processed for user `fc457fd883664398bdfe033e8fd8c02b`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:33.916675 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9d7e556e-1505-49c8-92fd-ff34b308f51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:33.991952 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:34.027022 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 964/1951] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:52:34.068148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316df66-dd96-4e62-aab7-cc509002a242 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:34.071935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e316df66-dd96-4e62-aab7-cc509002a242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:34.239687 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316df66-dd96-4e62-aab7-cc509002a242 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:34.250661 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7d7b1-6ee5-4a7c-a75b-55c54713740e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:34.252719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7d7b1-6ee5-4a7c-a75b-55c54713740e service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:34.253108 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7d7b1-6ee5-4a7c-a75b-55c54713740e service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:34.253356 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7d7b1-6ee5-4a7c-a75b-55c54713740e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:34.281695 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c7d7b1-6ee5-4a7c-a75b-55c54713740e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:34.283577 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c7d7b1-6ee5-4a7c-a75b-55c54713740e service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:34.321102 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 988/1952] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:34 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:34.767241 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7e556e-1505-49c8-92fd-ff34b308f51e None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:34.772314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9d7e556e-1505-49c8-92fd-ff34b308f51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:34.969669 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7e556e-1505-49c8-92fd-ff34b308f51e None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:34.984440 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a02f0-46d6-4fa0-b89a-aeec8f09c05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:34.986333 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a02f0-46d6-4fa0-b89a-aeec8f09c05b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:34.987049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a02f0-46d6-4fa0-b89a-aeec8f09c05b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:34.987518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a02f0-46d6-4fa0-b89a-aeec8f09c05b admin admin] PATH_INFO: `/v3/projects/2c6ba6c6a3c242d9936865a71c167c1d` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:34.989716 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a02f0-46d6-4fa0-b89a-aeec8f09c05b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c6ba6c6a3c242d9936865a71c167c1d)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:34.993273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a02f0-46d6-4fa0-b89a-aeec8f09c05b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:35.097166 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-dd6a02f0-46d6-4fa0-b89a-aeec8f09c05b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c6ba6c6a3c242d9936865a71c167c1d', 'request_id': 'req-dd6a02f0-46d6-4fa0-b89a-aeec8f09c05b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:35.114201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-dd6a02f0-46d6-4fa0-b89a-aeec8f09c05b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c6ba6c6a3c242d9936865a71c167c1d', 'request_id': 'req-dd6a02f0-46d6-4fa0-b89a-aeec8f09c05b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:35.134389 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-dd6a02f0-46d6-4fa0-b89a-aeec8f09c05b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c6ba6c6a3c242d9936865a71c167c1d', 'request_id': 'req-dd6a02f0-46d6-4fa0-b89a-aeec8f09c05b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:35.175958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-dd6a02f0-46d6-4fa0-b89a-aeec8f09c05b admin admin] The token cache is being invalidate because project 2c6ba6c6a3c242d9936865a71c167c1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:52:35.176437 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-dd6a02f0-46d6-4fa0-b89a-aeec8f09c05b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:35.176970 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-dd6a02f0-46d6-4fa0-b89a-aeec8f09c05b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:35.179598 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 965/1953] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:52:34 2022] DELETE /identity/v3/projects/2c6ba6c6a3c242d9936865a71c167c1d => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:35.187277 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108626b-7c77-47df-858a-98af80465995 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:35.188050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108626b-7c77-47df-858a-98af80465995 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:35.188050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108626b-7c77-47df-858a-98af80465995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:35.236895 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e108626b-7c77-47df-858a-98af80465995 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:35.241383 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e108626b-7c77-47df-858a-98af80465995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:35.313642 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:35.355595 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 989/1954] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:35.367629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f716db-2cd3-4749-86f4-a302cd656868 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:35.367915 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f716db-2cd3-4749-86f4-a302cd656868 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:35.368201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f716db-2cd3-4749-86f4-a302cd656868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:35.391546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-a0f716db-2cd3-4749-86f4-a302cd656868 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:35.392781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a0f716db-2cd3-4749-86f4-a302cd656868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:35.402903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 966/1955] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:52:35.408352 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108626b-7c77-47df-858a-98af80465995 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:35.420635 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108626b-7c77-47df-858a-98af80465995 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:35.426794 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0b7a7-fcc8-4944-9f22-2b50f2351caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:35.427527 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0b7a7-fcc8-4944-9f22-2b50f2351caa admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:35.427816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0b7a7-fcc8-4944-9f22-2b50f2351caa admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:35.428092 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0b7a7-fcc8-4944-9f22-2b50f2351caa admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:35.428758 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0b7a7-fcc8-4944-9f22-2b50f2351caa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:52:35.429088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0b7a7-fcc8-4944-9f22-2b50f2351caa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:35.430784 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0b7a7-fcc8-4944-9f22-2b50f2351caa admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:35.459208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 990/1956] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:52:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:52:35.461160 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f716db-2cd3-4749-86f4-a302cd656868 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:35.473164 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f716db-2cd3-4749-86f4-a302cd656868 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:35.479889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54805189-7140-4b75-8625-6038b3d99b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:35.480487 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54805189-7140-4b75-8625-6038b3d99b29 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:35.480787 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54805189-7140-4b75-8625-6038b3d99b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:35.481055 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54805189-7140-4b75-8625-6038b3d99b29 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:35.481810 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54805189-7140-4b75-8625-6038b3d99b29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:35.484117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54805189-7140-4b75-8625-6038b3d99b29 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:35.528029 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 967/1957] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:52:35 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:52:35.533935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0b7a7-fcc8-4944-9f22-2b50f2351caa admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:35.549157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0b7a7-fcc8-4944-9f22-2b50f2351caa admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:35.640030 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502cf24-4505-4463-a00a-24ea5e94794d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:35.642604 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502cf24-4505-4463-a00a-24ea5e94794d admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:35.643145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502cf24-4505-4463-a00a-24ea5e94794d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:35.643621 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502cf24-4505-4463-a00a-24ea5e94794d admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:35.644787 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9502cf24-4505-4463-a00a-24ea5e94794d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:35.648624 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9502cf24-4505-4463-a00a-24ea5e94794d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:35.720859 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 991/1958] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:52:35 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:35.726667 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54805189-7140-4b75-8625-6038b3d99b29 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:35.746092 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54805189-7140-4b75-8625-6038b3d99b29 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:35.760673 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9eb09a-6529-46ca-94af-1316b7df9e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:35.761817 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9eb09a-6529-46ca-94af-1316b7df9e0a admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:35.762597 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9eb09a-6529-46ca-94af-1316b7df9e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:35.763150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9eb09a-6529-46ca-94af-1316b7df9e0a admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:35.764508 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9eb09a-6529-46ca-94af-1316b7df9e0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:35.768188 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9eb09a-6529-46ca-94af-1316b7df9e0a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:35.790652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 968/1959] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 08:52:35.791561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502cf24-4505-4463-a00a-24ea5e94794d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:35.805737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502cf24-4505-4463-a00a-24ea5e94794d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:35.812855 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870226c0-eb3f-477b-8e11-28bb838a6a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:35.813617 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870226c0-eb3f-477b-8e11-28bb838a6a5f admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:35.813872 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870226c0-eb3f-477b-8e11-28bb838a6a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:35.814155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870226c0-eb3f-477b-8e11-28bb838a6a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:35.814837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870226c0-eb3f-477b-8e11-28bb838a6a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:35.816831 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870226c0-eb3f-477b-8e11-28bb838a6a5f admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:35.845918 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 992/1960] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:35.849097 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9eb09a-6529-46ca-94af-1316b7df9e0a admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:35.865505 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9eb09a-6529-46ca-94af-1316b7df9e0a admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:35.873240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd73d4c-e079-406f-8400-7240822b7af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:35.874073 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd73d4c-e079-406f-8400-7240822b7af1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:35.874421 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd73d4c-e079-406f-8400-7240822b7af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:35.874706 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd73d4c-e079-406f-8400-7240822b7af1 admin admin] PATH_INFO: `/v3/projects/1a93639f05644b30aa8ff9e9c2078126/users/4f9a447a8d46446fa1eedda883212be9/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:35.911918 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd73d4c-e079-406f-8400-7240822b7af1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a93639f05644b30aa8ff9e9c2078126, user_id=4f9a447a8d46446fa1eedda883212be9, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:35.916200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd73d4c-e079-406f-8400-7240822b7af1 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:35.942961 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 969/1961] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:35 2022] PUT /identity/v3/projects/1a93639f05644b30aa8ff9e9c2078126/users/4f9a447a8d46446fa1eedda883212be9/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:52:35.955051 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f7abd-c657-48ed-9ae2-6a7e443509d0 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:35.955475 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f7abd-c657-48ed-9ae2-6a7e443509d0 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:35.955881 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f7abd-c657-48ed-9ae2-6a7e443509d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:36.003103 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e63f7abd-c657-48ed-9ae2-6a7e443509d0 None None] MFA Rules not processed for user `4f9a447a8d46446fa1eedda883212be9`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:36.005784 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e63f7abd-c657-48ed-9ae2-6a7e443509d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:36.187118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 993/1962] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:36.190640 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd73d4c-e079-406f-8400-7240822b7af1 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:36.217404 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:36.384184 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd73d4c-e079-406f-8400-7240822b7af1 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:36.401737 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770d47e-b05c-493a-a5d9-b5ddd19f878c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:36.402329 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770d47e-b05c-493a-a5d9-b5ddd19f878c admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:36.402617 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770d47e-b05c-493a-a5d9-b5ddd19f878c admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:36.402934 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770d47e-b05c-493a-a5d9-b5ddd19f878c admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:36.403817 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9770d47e-b05c-493a-a5d9-b5ddd19f878c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:36.405979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9770d47e-b05c-493a-a5d9-b5ddd19f878c admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:36.481386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 970/1963] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:52:36 2022] POST /identity/v3/projects => generated 385 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:36.482420 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f7abd-c657-48ed-9ae2-6a7e443509d0 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:36.506759 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f7abd-c657-48ed-9ae2-6a7e443509d0 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:36.599546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:36.688365 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a938ed8-90c5-45f0-bd48-36dcb5604e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:36.690109 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a938ed8-90c5-45f0-bd48-36dcb5604e0d admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:36.690413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a938ed8-90c5-45f0-bd48-36dcb5604e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:36.690779 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a938ed8-90c5-45f0-bd48-36dcb5604e0d admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:36.691665 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a938ed8-90c5-45f0-bd48-36dcb5604e0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:36.695642 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a938ed8-90c5-45f0-bd48-36dcb5604e0d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:36.753196 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770d47e-b05c-493a-a5d9-b5ddd19f878c admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:36.753681 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 994/1964] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:52:36 2022] POST /identity/v3/users => generated 299 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:52:36.777510 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770d47e-b05c-493a-a5d9-b5ddd19f878c admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:36.795012 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003740d-9a78-49ba-8c71-cb7df3c66bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:36.796447 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003740d-9a78-49ba-8c71-cb7df3c66bcf admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:36.796876 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003740d-9a78-49ba-8c71-cb7df3c66bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:36.797192 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003740d-9a78-49ba-8c71-cb7df3c66bcf admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:36.797883 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003740d-9a78-49ba-8c71-cb7df3c66bcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:36.799873 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003740d-9a78-49ba-8c71-cb7df3c66bcf admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:36.858786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a938ed8-90c5-45f0-bd48-36dcb5604e0d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:36.875649 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 971/1965] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:36 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:36.878368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a938ed8-90c5-45f0-bd48-36dcb5604e0d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:36.889279 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30486647-e06f-4169-a72f-91ace6e4c9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:36.890496 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30486647-e06f-4169-a72f-91ace6e4c9f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:36.893264 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30486647-e06f-4169-a72f-91ace6e4c9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:36.894130 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30486647-e06f-4169-a72f-91ace6e4c9f3 admin admin] PATH_INFO: `/v3/projects/bbef6d692ab04ff4998d5f1564e3634f/users/2c72c60887114c2eb6eff02d223f6355/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:36.944046 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30486647-e06f-4169-a72f-91ace6e4c9f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbef6d692ab04ff4998d5f1564e3634f, user_id=2c72c60887114c2eb6eff02d223f6355, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:36.954804 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30486647-e06f-4169-a72f-91ace6e4c9f3 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:36.956051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003740d-9a78-49ba-8c71-cb7df3c66bcf admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:36.957902 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7003740d-9a78-49ba-8c71-cb7df3c66bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:36.976010 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 995/1966] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:36 2022] PUT /identity/v3/projects/bbef6d692ab04ff4998d5f1564e3634f/users/2c72c60887114c2eb6eff02d223f6355/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:36.980331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30486647-e06f-4169-a72f-91ace6e4c9f3 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:36.984446 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003740d-9a78-49ba-8c71-cb7df3c66bcf admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:37.091979 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30486647-e06f-4169-a72f-91ace6e4c9f3 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:37.097904 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ebfb6-f74f-4b28-9efc-c06a5213e346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:37.098733 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ebfb6-f74f-4b28-9efc-c06a5213e346 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:37.099144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ebfb6-f74f-4b28-9efc-c06a5213e346 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:37.099656 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ebfb6-f74f-4b28-9efc-c06a5213e346 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:37.100445 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26ebfb6-f74f-4b28-9efc-c06a5213e346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:37.102217 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26ebfb6-f74f-4b28-9efc-c06a5213e346 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:37.119437 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 996/1967] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:36 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 08:52:37.119690 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95085a76-7b38-41d4-801c-147d8e8fc359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:37.119690 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95085a76-7b38-41d4-801c-147d8e8fc359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:37.119690 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95085a76-7b38-41d4-801c-147d8e8fc359 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:37.119690 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95085a76-7b38-41d4-801c-147d8e8fc359 admin admin] PATH_INFO: `/v3/users/c74fce0dff824bdc8e8013278d8eeb80` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:37.120338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95085a76-7b38-41d4-801c-147d8e8fc359 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c74fce0dff824bdc8e8013278d8eeb80)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:37.120929 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ebfb6-f74f-4b28-9efc-c06a5213e346 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:37.122566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95085a76-7b38-41d4-801c-147d8e8fc359 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:37.132897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ebfb6-f74f-4b28-9efc-c06a5213e346 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:37.140424 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce4522-eaf2-42e1-bf3b-91e4ccaec127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:37.142629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce4522-eaf2-42e1-bf3b-91e4ccaec127 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:37.143258 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce4522-eaf2-42e1-bf3b-91e4ccaec127 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:37.144052 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce4522-eaf2-42e1-bf3b-91e4ccaec127 admin admin] PATH_INFO: `/v3/projects/bbef6d692ab04ff4998d5f1564e3634f/users/2c72c60887114c2eb6eff02d223f6355/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:37.151263 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce4522-eaf2-42e1-bf3b-91e4ccaec127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbef6d692ab04ff4998d5f1564e3634f, user_id=2c72c60887114c2eb6eff02d223f6355, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:37.155100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce4522-eaf2-42e1-bf3b-91e4ccaec127 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:37.169830 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 997/1968] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:37 2022] PUT /identity/v3/projects/bbef6d692ab04ff4998d5f1564e3634f/users/2c72c60887114c2eb6eff02d223f6355/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:37.185109 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a9bfc-a5b8-4246-be96-8e30cb82754e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:37.185652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a9bfc-a5b8-4246-be96-8e30cb82754e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:37.185808 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a9bfc-a5b8-4246-be96-8e30cb82754e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:37.207011 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-95085a76-7b38-41d4-801c-147d8e8fc359 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c74fce0dff824bdc8e8013278d8eeb80'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:37.221264 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-95085a76-7b38-41d4-801c-147d8e8fc359 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c74fce0dff824bdc8e8013278d8eeb80'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:37.226707 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-d79a9bfc-a5b8-4246-be96-8e30cb82754e None None] MFA Rules not processed for user `2c72c60887114c2eb6eff02d223f6355`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:37.228050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d79a9bfc-a5b8-4246-be96-8e30cb82754e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:37.233029 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-95085a76-7b38-41d4-801c-147d8e8fc359 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c74fce0dff824bdc8e8013278d8eeb80'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:37.234331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-95085a76-7b38-41d4-801c-147d8e8fc359 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c74fce0dff824bdc8e8013278d8eeb80'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:37.235551 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-95085a76-7b38-41d4-801c-147d8e8fc359 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c74fce0dff824bdc8e8013278d8eeb80'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:37.262426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 972/1969] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:52:36 2022] DELETE /identity/v3/users/c74fce0dff824bdc8e8013278d8eeb80 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:52:37.269153 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4b28a-7d95-4eb8-b0a0-7b5ebf98b545 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:37.269863 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4b28a-7d95-4eb8-b0a0-7b5ebf98b545 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:37.270307 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4b28a-7d95-4eb8-b0a0-7b5ebf98b545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:37.295353 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:37.303893 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-60a4b28a-7d95-4eb8-b0a0-7b5ebf98b545 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:37.305651 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-60a4b28a-7d95-4eb8-b0a0-7b5ebf98b545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:37.342817 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 998/1970] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:37 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:52:37.357495 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a9bfc-a5b8-4246-be96-8e30cb82754e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:37.361741 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d79a9bfc-a5b8-4246-be96-8e30cb82754e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:37.385013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:37.423956 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 973/1971] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:52:37.431805 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4b28a-7d95-4eb8-b0a0-7b5ebf98b545 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:37.434236 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-60a4b28a-7d95-4eb8-b0a0-7b5ebf98b545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:37.486210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a9bfc-a5b8-4246-be96-8e30cb82754e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:37.541391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4b28a-7d95-4eb8-b0a0-7b5ebf98b545 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:37.547965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbe57a-d3eb-4536-b37d-835d5ca95a31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:37.549073 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbe57a-d3eb-4536-b37d-835d5ca95a31 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:37.549550 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbe57a-d3eb-4536-b37d-835d5ca95a31 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:37.550016 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbe57a-d3eb-4536-b37d-835d5ca95a31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:37.551376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e96ac7-918d-408a-bb45-0ebb75da7d30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:37.552028 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e96ac7-918d-408a-bb45-0ebb75da7d30 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:37.552304 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e96ac7-918d-408a-bb45-0ebb75da7d30 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:37.552631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e96ac7-918d-408a-bb45-0ebb75da7d30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:37.555603 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a0e96ac7-918d-408a-bb45-0ebb75da7d30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:37.567967 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bbe57a-d3eb-4536-b37d-835d5ca95a31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:37.570820 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bbe57a-d3eb-4536-b37d-835d5ca95a31 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:37.587213 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 974/1972] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:52:37.652729 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e96ac7-918d-408a-bb45-0ebb75da7d30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:37.655689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e96ac7-918d-408a-bb45-0ebb75da7d30 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:37.672638 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 999/1973] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:37.764531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbe57a-d3eb-4536-b37d-835d5ca95a31 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:37.766653 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-98bbe57a-d3eb-4536-b37d-835d5ca95a31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:37.782769 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbe57a-d3eb-4536-b37d-835d5ca95a31 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:37.789540 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b9803-78ce-4f2e-b151-cd39dd8a87be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:37.790711 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b9803-78ce-4f2e-b151-cd39dd8a87be admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:37.791297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b9803-78ce-4f2e-b151-cd39dd8a87be admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:37.791786 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b9803-78ce-4f2e-b151-cd39dd8a87be admin admin] PATH_INFO: `/v3/users/f7ba7907de034abfa0a85cff36821812` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:37.794439 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b9803-78ce-4f2e-b151-cd39dd8a87be admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7ba7907de034abfa0a85cff36821812)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:37.798495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b9803-78ce-4f2e-b151-cd39dd8a87be admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:37.921707 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0b9b9803-78ce-4f2e-b151-cd39dd8a87be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7ba7907de034abfa0a85cff36821812'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:37.935825 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0b9b9803-78ce-4f2e-b151-cd39dd8a87be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7ba7907de034abfa0a85cff36821812'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:37.954007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0b9b9803-78ce-4f2e-b151-cd39dd8a87be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7ba7907de034abfa0a85cff36821812'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:37.955316 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0b9b9803-78ce-4f2e-b151-cd39dd8a87be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7ba7907de034abfa0a85cff36821812'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:37.956341 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0b9b9803-78ce-4f2e-b151-cd39dd8a87be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7ba7907de034abfa0a85cff36821812'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:38.011484 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 975/1974] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:52:37 2022] DELETE /identity/v3/users/f7ba7907de034abfa0a85cff36821812 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:52:38.017725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0bfc5-8910-4fe1-a16c-5bdb47476290 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:38.019186 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0bfc5-8910-4fe1-a16c-5bdb47476290 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:38.019896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0bfc5-8910-4fe1-a16c-5bdb47476290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:38.074333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-04d0bfc5-8910-4fe1-a16c-5bdb47476290 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:38.076641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-04d0bfc5-8910-4fe1-a16c-5bdb47476290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:38.244360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1000/1975] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:38.261755 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b9803-78ce-4f2e-b151-cd39dd8a87be admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:38.264017 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0b9b9803-78ce-4f2e-b151-cd39dd8a87be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:38.351347 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:38.358694 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0bfc5-8910-4fe1-a16c-5bdb47476290 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:38.360930 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-04d0bfc5-8910-4fe1-a16c-5bdb47476290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:38.382121 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0bfc5-8910-4fe1-a16c-5bdb47476290 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:38.404292 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3c837-dc41-49e5-8c83-206b860f5bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:38.406149 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3c837-dc41-49e5-8c83-206b860f5bf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:38.406852 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3c837-dc41-49e5-8c83-206b860f5bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:38.408582 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3c837-dc41-49e5-8c83-206b860f5bf8 admin admin] PATH_INFO: `/v3/projects/a3d411aca75d49989de42d3e907f25e9` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:38.411824 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3c837-dc41-49e5-8c83-206b860f5bf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3d411aca75d49989de42d3e907f25e9)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:38.421548 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3c837-dc41-49e5-8c83-206b860f5bf8 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:38.486419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:38.492117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b9803-78ce-4f2e-b151-cd39dd8a87be admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:38.512125 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cef9fc-4e11-41e5-98a4-0d92e44a9915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:38.512851 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cef9fc-4e11-41e5-98a4-0d92e44a9915 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:38.513138 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cef9fc-4e11-41e5-98a4-0d92e44a9915 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:38.513421 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cef9fc-4e11-41e5-98a4-0d92e44a9915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:38.546087 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cef9fc-4e11-41e5-98a4-0d92e44a9915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:38.549683 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cef9fc-4e11-41e5-98a4-0d92e44a9915 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:38.593832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 976/1976] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:38.617275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-15e3c837-dc41-49e5-8c83-206b860f5bf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3d411aca75d49989de42d3e907f25e9', 'request_id': 'req-15e3c837-dc41-49e5-8c83-206b860f5bf8'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:38.626975 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-15e3c837-dc41-49e5-8c83-206b860f5bf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3d411aca75d49989de42d3e907f25e9', 'request_id': 'req-15e3c837-dc41-49e5-8c83-206b860f5bf8'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:38.640225 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-15e3c837-dc41-49e5-8c83-206b860f5bf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3d411aca75d49989de42d3e907f25e9', 'request_id': 'req-15e3c837-dc41-49e5-8c83-206b860f5bf8'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:38.661460 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-15e3c837-dc41-49e5-8c83-206b860f5bf8 admin admin] The token cache is being invalidate because project a3d411aca75d49989de42d3e907f25e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:52:38.662057 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-15e3c837-dc41-49e5-8c83-206b860f5bf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:38.663374 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-15e3c837-dc41-49e5-8c83-206b860f5bf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:38.673000 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cef9fc-4e11-41e5-98a4-0d92e44a9915 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:38.674973 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1001/1977] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:52:38 2022] DELETE /identity/v3/projects/a3d411aca75d49989de42d3e907f25e9 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:38.698012 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cef9fc-4e11-41e5-98a4-0d92e44a9915 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:38.808593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860a9b0-acae-433c-aed7-23fbff0f57a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:38.809685 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860a9b0-acae-433c-aed7-23fbff0f57a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:38.809685 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860a9b0-acae-433c-aed7-23fbff0f57a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:38.809966 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860a9b0-acae-433c-aed7-23fbff0f57a2 admin admin] PATH_INFO: `/v3/users/db512ee6de9944669d9eaf2b5ae610e7` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:38.811283 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860a9b0-acae-433c-aed7-23fbff0f57a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db512ee6de9944669d9eaf2b5ae610e7)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:38.813936 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860a9b0-acae-433c-aed7-23fbff0f57a2 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:38.904759 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c860a9b0-acae-433c-aed7-23fbff0f57a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db512ee6de9944669d9eaf2b5ae610e7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:38.919166 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c860a9b0-acae-433c-aed7-23fbff0f57a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db512ee6de9944669d9eaf2b5ae610e7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:38.931725 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c860a9b0-acae-433c-aed7-23fbff0f57a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db512ee6de9944669d9eaf2b5ae610e7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:38.933271 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c860a9b0-acae-433c-aed7-23fbff0f57a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db512ee6de9944669d9eaf2b5ae610e7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:38.934345 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c860a9b0-acae-433c-aed7-23fbff0f57a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db512ee6de9944669d9eaf2b5ae610e7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:38.978041 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 977/1978] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:52:38 2022] DELETE /identity/v3/users/db512ee6de9944669d9eaf2b5ae610e7 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:39.164358 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3c837-dc41-49e5-8c83-206b860f5bf8 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:39.166801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-15e3c837-dc41-49e5-8c83-206b860f5bf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:39.188318 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3c837-dc41-49e5-8c83-206b860f5bf8 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:39.339943 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57ab03-daca-40e3-8b1b-44a6b0112a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:39.341172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57ab03-daca-40e3-8b1b-44a6b0112a56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:39.342067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57ab03-daca-40e3-8b1b-44a6b0112a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:39.343428 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57ab03-daca-40e3-8b1b-44a6b0112a56 admin admin] PATH_INFO: `/v3/projects/4781b11c08544fb481a5126565bb9bee` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:39.346001 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57ab03-daca-40e3-8b1b-44a6b0112a56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4781b11c08544fb481a5126565bb9bee)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:39.351171 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57ab03-daca-40e3-8b1b-44a6b0112a56 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:39.470653 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2f57ab03-daca-40e3-8b1b-44a6b0112a56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4781b11c08544fb481a5126565bb9bee', 'request_id': 'req-2f57ab03-daca-40e3-8b1b-44a6b0112a56'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:39.486534 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2f57ab03-daca-40e3-8b1b-44a6b0112a56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4781b11c08544fb481a5126565bb9bee', 'request_id': 'req-2f57ab03-daca-40e3-8b1b-44a6b0112a56'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:39.504868 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2f57ab03-daca-40e3-8b1b-44a6b0112a56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4781b11c08544fb481a5126565bb9bee', 'request_id': 'req-2f57ab03-daca-40e3-8b1b-44a6b0112a56'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:39.549729 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2f57ab03-daca-40e3-8b1b-44a6b0112a56 admin admin] The token cache is being invalidate because project 4781b11c08544fb481a5126565bb9bee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:52:39.550300 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2f57ab03-daca-40e3-8b1b-44a6b0112a56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:39.551644 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2f57ab03-daca-40e3-8b1b-44a6b0112a56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:39.560016 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1002/1979] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:52:39 2022] DELETE /identity/v3/projects/4781b11c08544fb481a5126565bb9bee => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:39.561382 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860a9b0-acae-433c-aed7-23fbff0f57a2 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:39.563148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c860a9b0-acae-433c-aed7-23fbff0f57a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:39.599633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860a9b0-acae-433c-aed7-23fbff0f57a2 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:39.655319 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57ab03-daca-40e3-8b1b-44a6b0112a56 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:39.677405 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2f57ab03-daca-40e3-8b1b-44a6b0112a56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:39.743184 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:39.749638 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:39.779912 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ffc285-ebc9-4ce4-a280-f5c84a19afdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:39.780984 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ffc285-ebc9-4ce4-a280-f5c84a19afdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:39.781620 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ffc285-ebc9-4ce4-a280-f5c84a19afdf admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:39.781809 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ffc285-ebc9-4ce4-a280-f5c84a19afdf admin admin] PATH_INFO: `/v3/users/20bcc74e24f14c57a2cbd31ebb93db5c` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:39.783799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ffc285-ebc9-4ce4-a280-f5c84a19afdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=20bcc74e24f14c57a2cbd31ebb93db5c)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:39.787458 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ffc285-ebc9-4ce4-a280-f5c84a19afdf admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:39.823564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57ab03-daca-40e3-8b1b-44a6b0112a56 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:39.832316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b1d06-d427-462e-b804-d8f47944ce4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:39.833637 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b1d06-d427-462e-b804-d8f47944ce4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:39.834176 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b1d06-d427-462e-b804-d8f47944ce4e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:39.834618 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b1d06-d427-462e-b804-d8f47944ce4e admin admin] PATH_INFO: `/v3/projects/42e46784f64441c99a3fa30526655d5a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:39.836817 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b1d06-d427-462e-b804-d8f47944ce4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=42e46784f64441c99a3fa30526655d5a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:39.841034 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b1d06-d427-462e-b804-d8f47944ce4e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:39.878394 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-42ffc285-ebc9-4ce4-a280-f5c84a19afdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20bcc74e24f14c57a2cbd31ebb93db5c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:39.892513 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-42ffc285-ebc9-4ce4-a280-f5c84a19afdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20bcc74e24f14c57a2cbd31ebb93db5c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:39.903075 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-42ffc285-ebc9-4ce4-a280-f5c84a19afdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20bcc74e24f14c57a2cbd31ebb93db5c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:39.903932 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-42ffc285-ebc9-4ce4-a280-f5c84a19afdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20bcc74e24f14c57a2cbd31ebb93db5c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:39.904711 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-42ffc285-ebc9-4ce4-a280-f5c84a19afdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20bcc74e24f14c57a2cbd31ebb93db5c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:39.929871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 978/1980] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:52:39 2022] DELETE /identity/v3/users/20bcc74e24f14c57a2cbd31ebb93db5c => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:52:39.936067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c53b1d06-d427-462e-b804-d8f47944ce4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42e46784f64441c99a3fa30526655d5a', 'request_id': 'req-c53b1d06-d427-462e-b804-d8f47944ce4e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:39.951477 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c53b1d06-d427-462e-b804-d8f47944ce4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42e46784f64441c99a3fa30526655d5a', 'request_id': 'req-c53b1d06-d427-462e-b804-d8f47944ce4e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:39.967694 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c53b1d06-d427-462e-b804-d8f47944ce4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42e46784f64441c99a3fa30526655d5a', 'request_id': 'req-c53b1d06-d427-462e-b804-d8f47944ce4e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:40.010929 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c53b1d06-d427-462e-b804-d8f47944ce4e admin admin] The token cache is being invalidate because project 42e46784f64441c99a3fa30526655d5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:52:40.010929 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c53b1d06-d427-462e-b804-d8f47944ce4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:40.010929 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c53b1d06-d427-462e-b804-d8f47944ce4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:40.015907 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1003/1981] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:52:39 2022] DELETE /identity/v3/projects/42e46784f64441c99a3fa30526655d5a => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:40.017035 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16915386-79fd-45c3-813b-341b7504e03b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:40.017423 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16915386-79fd-45c3-813b-341b7504e03b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:40.017801 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16915386-79fd-45c3-813b-341b7504e03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:40.045749 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-16915386-79fd-45c3-813b-341b7504e03b None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:40.048801 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-16915386-79fd-45c3-813b-341b7504e03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:40.202861 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 979/1982] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:52:40.213935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082ee30-bbbe-4b05-8f82-0bd9aab60ecd None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:40.214425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082ee30-bbbe-4b05-8f82-0bd9aab60ecd None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:40.214861 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082ee30-bbbe-4b05-8f82-0bd9aab60ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:40.266698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-d082ee30-bbbe-4b05-8f82-0bd9aab60ecd None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:40.267604 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d082ee30-bbbe-4b05-8f82-0bd9aab60ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:40.282921 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16915386-79fd-45c3-813b-341b7504e03b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:40.288166 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1004/1983] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:40.305778 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16915386-79fd-45c3-813b-341b7504e03b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:40.320627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36976d40-5195-4507-82b8-f93748817e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:40.320627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36976d40-5195-4507-82b8-f93748817e61 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:40.320987 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36976d40-5195-4507-82b8-f93748817e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:40.320987 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36976d40-5195-4507-82b8-f93748817e61 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:40.321721 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36976d40-5195-4507-82b8-f93748817e61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:52:40.322038 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36976d40-5195-4507-82b8-f93748817e61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:40.331207 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36976d40-5195-4507-82b8-f93748817e61 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:40.380013 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 980/1984] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:52:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:52:40.384839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082ee30-bbbe-4b05-8f82-0bd9aab60ecd None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:40.397512 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082ee30-bbbe-4b05-8f82-0bd9aab60ecd None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:40.416313 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d9f8f-1faf-482f-82c3-996429e56b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:40.418339 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d9f8f-1faf-482f-82c3-996429e56b23 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:40.418722 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d9f8f-1faf-482f-82c3-996429e56b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:40.420323 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d9f8f-1faf-482f-82c3-996429e56b23 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:40.421939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d9f8f-1faf-482f-82c3-996429e56b23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:40.424956 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d9f8f-1faf-482f-82c3-996429e56b23 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:40.483958 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1005/1985] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:52:40 2022] POST /identity/v3/projects => generated 367 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:52:40.488136 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36976d40-5195-4507-82b8-f93748817e61 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:40.500347 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36976d40-5195-4507-82b8-f93748817e61 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:40.580535 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:40.596575 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e01eb-058d-4673-a7ef-b05b2bb2651d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:40.597609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e01eb-058d-4673-a7ef-b05b2bb2651d admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:40.598040 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e01eb-058d-4673-a7ef-b05b2bb2651d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:40.598687 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e01eb-058d-4673-a7ef-b05b2bb2651d admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:40.599526 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e01eb-058d-4673-a7ef-b05b2bb2651d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:40.602339 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e01eb-058d-4673-a7ef-b05b2bb2651d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:40.634653 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 981/1986] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:52:40 2022] POST /identity/v3/users => generated 284 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:52:40.639508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d9f8f-1faf-482f-82c3-996429e56b23 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:40.650796 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d9f8f-1faf-482f-82c3-996429e56b23 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:40.656487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8d3d3-3bdc-4083-8859-c63bf539b98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:40.659143 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8d3d3-3bdc-4083-8859-c63bf539b98a admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:40.659430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8d3d3-3bdc-4083-8859-c63bf539b98a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:40.659684 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8d3d3-3bdc-4083-8859-c63bf539b98a admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:40.660393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8d3d3-3bdc-4083-8859-c63bf539b98a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:40.662090 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8d3d3-3bdc-4083-8859-c63bf539b98a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:40.674499 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1006/1987] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:40.679933 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e01eb-058d-4673-a7ef-b05b2bb2651d admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:40.703090 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e01eb-058d-4673-a7ef-b05b2bb2651d admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:40.717754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0046b-47f2-40c4-bc92-e63c5079768f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:40.718704 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0046b-47f2-40c4-bc92-e63c5079768f admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:40.718878 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0046b-47f2-40c4-bc92-e63c5079768f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:40.719175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0046b-47f2-40c4-bc92-e63c5079768f admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:40.727660 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f0046b-47f2-40c4-bc92-e63c5079768f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:40.742091 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f0046b-47f2-40c4-bc92-e63c5079768f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:40.771221 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 982/1988] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:40.790729 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8d3d3-3bdc-4083-8859-c63bf539b98a admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:40.823480 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8d3d3-3bdc-4083-8859-c63bf539b98a admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:40.837859 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a5800d-d3f5-4fe8-a786-6a4ec14c3405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:40.838726 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5800d-d3f5-4fe8-a786-6a4ec14c3405 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:40.839074 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5800d-d3f5-4fe8-a786-6a4ec14c3405 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:40.839427 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a5800d-d3f5-4fe8-a786-6a4ec14c3405 admin admin] PATH_INFO: `/v3/projects/9b008635fdf24cc7b5cceb0ceab1e75c/users/644e9bbd92e14a88b4c5b91712b2e671/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:40.864583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a5800d-d3f5-4fe8-a786-6a4ec14c3405 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b008635fdf24cc7b5cceb0ceab1e75c, user_id=644e9bbd92e14a88b4c5b91712b2e671, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:40.868151 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a5800d-d3f5-4fe8-a786-6a4ec14c3405 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:40.901334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1007/1989] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:40 2022] PUT /identity/v3/projects/9b008635fdf24cc7b5cceb0ceab1e75c/users/644e9bbd92e14a88b4c5b91712b2e671/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:40.903447 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961dd4ca-0c03-4cae-9529-6bb0daa55da8 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:40.904018 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961dd4ca-0c03-4cae-9529-6bb0daa55da8 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:40.904018 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961dd4ca-0c03-4cae-9529-6bb0daa55da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:40.944983 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-961dd4ca-0c03-4cae-9529-6bb0daa55da8 None None] MFA Rules not processed for user `644e9bbd92e14a88b4c5b91712b2e671`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:40.947751 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-961dd4ca-0c03-4cae-9529-6bb0daa55da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:41.055620 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a5800d-d3f5-4fe8-a786-6a4ec14c3405 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:41.057492 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-57a5800d-d3f5-4fe8-a786-6a4ec14c3405 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:41.138309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 983/1990] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:41.143255 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961dd4ca-0c03-4cae-9529-6bb0daa55da8 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:41.235072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a5800d-d3f5-4fe8-a786-6a4ec14c3405 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:41.251439 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e3dcd-9a85-40f1-86aa-b35b082493ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:41.256310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e3dcd-9a85-40f1-86aa-b35b082493ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:41.257009 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e3dcd-9a85-40f1-86aa-b35b082493ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:41.257560 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e3dcd-9a85-40f1-86aa-b35b082493ea admin admin] PATH_INFO: `/v3/projects/1b1a83e28df941a5b94aab9fafe23cf9` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:41.265571 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e3dcd-9a85-40f1-86aa-b35b082493ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b1a83e28df941a5b94aab9fafe23cf9)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:41.269876 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961dd4ca-0c03-4cae-9529-6bb0daa55da8 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:41.273703 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e3dcd-9a85-40f1-86aa-b35b082493ea admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:41.275150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6124f-1ca6-4fea-bfc2-ac3d2365b31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:41.275969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6124f-1ca6-4fea-bfc2-ac3d2365b31d admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:41.276299 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6124f-1ca6-4fea-bfc2-ac3d2365b31d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:41.276583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6124f-1ca6-4fea-bfc2-ac3d2365b31d admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:41.277242 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6124f-1ca6-4fea-bfc2-ac3d2365b31d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:41.279359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6124f-1ca6-4fea-bfc2-ac3d2365b31d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:41.291846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:41.348722 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 984/1991] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:52:41 2022] POST /identity/v3/projects => generated 369 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:41.358941 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6124f-1ca6-4fea-bfc2-ac3d2365b31d admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:41.467175 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-770e3dcd-9a85-40f1-86aa-b35b082493ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b1a83e28df941a5b94aab9fafe23cf9', 'request_id': 'req-770e3dcd-9a85-40f1-86aa-b35b082493ea'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:41.473849 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:41.478299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-770e3dcd-9a85-40f1-86aa-b35b082493ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b1a83e28df941a5b94aab9fafe23cf9', 'request_id': 'req-770e3dcd-9a85-40f1-86aa-b35b082493ea'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:41.493831 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-770e3dcd-9a85-40f1-86aa-b35b082493ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b1a83e28df941a5b94aab9fafe23cf9', 'request_id': 'req-770e3dcd-9a85-40f1-86aa-b35b082493ea'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:41.508673 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6124f-1ca6-4fea-bfc2-ac3d2365b31d admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:41.516055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-770e3dcd-9a85-40f1-86aa-b35b082493ea admin admin] The token cache is being invalidate because project 1b1a83e28df941a5b94aab9fafe23cf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:52:41.516638 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-770e3dcd-9a85-40f1-86aa-b35b082493ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:41.517811 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-770e3dcd-9a85-40f1-86aa-b35b082493ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:41.520928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1008/1992] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:52:41 2022] DELETE /identity/v3/projects/1b1a83e28df941a5b94aab9fafe23cf9 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:41.527136 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e3dcd-9a85-40f1-86aa-b35b082493ea admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:41.529604 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-770e3dcd-9a85-40f1-86aa-b35b082493ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:41.622580 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8286ee1-a9b1-47f4-86a8-cd78991fda4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:41.623613 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8286ee1-a9b1-47f4-86a8-cd78991fda4d admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:41.623995 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8286ee1-a9b1-47f4-86a8-cd78991fda4d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:41.624271 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8286ee1-a9b1-47f4-86a8-cd78991fda4d admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:41.625470 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8286ee1-a9b1-47f4-86a8-cd78991fda4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:41.628817 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8286ee1-a9b1-47f4-86a8-cd78991fda4d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:41.629545 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e3dcd-9a85-40f1-86aa-b35b082493ea admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:41.634904 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba412042-d4d0-4cbe-96bd-3e4a82e02364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:41.635578 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba412042-d4d0-4cbe-96bd-3e4a82e02364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:41.635859 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba412042-d4d0-4cbe-96bd-3e4a82e02364 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:41.636178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba412042-d4d0-4cbe-96bd-3e4a82e02364 admin admin] PATH_INFO: `/v3/users/dc5801e7e8e7478e9d03165ac63127f5` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:41.637559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba412042-d4d0-4cbe-96bd-3e4a82e02364 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc5801e7e8e7478e9d03165ac63127f5)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:41.639619 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba412042-d4d0-4cbe-96bd-3e4a82e02364 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:41.666947 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 985/1993] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:52:41 2022] POST /identity/v3/users => generated 291 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:52:41.668364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8286ee1-a9b1-47f4-86a8-cd78991fda4d admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:41.670318 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e8286ee1-a9b1-47f4-86a8-cd78991fda4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:41.684096 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8286ee1-a9b1-47f4-86a8-cd78991fda4d admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:41.689878 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde9fe7-4f6d-4781-91cd-21d139e7923b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:41.690786 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde9fe7-4f6d-4781-91cd-21d139e7923b admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:41.691000 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde9fe7-4f6d-4781-91cd-21d139e7923b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:41.691281 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde9fe7-4f6d-4781-91cd-21d139e7923b admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:41.692236 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dde9fe7-4f6d-4781-91cd-21d139e7923b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:41.693928 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dde9fe7-4f6d-4781-91cd-21d139e7923b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:41.710839 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 986/1994] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:41.711977 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde9fe7-4f6d-4781-91cd-21d139e7923b admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:41.719186 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-ba412042-d4d0-4cbe-96bd-3e4a82e02364 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc5801e7e8e7478e9d03165ac63127f5'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:41.723551 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde9fe7-4f6d-4781-91cd-21d139e7923b admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:41.729202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95015b63-ddb4-48f9-99ac-41d5e3dd601e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:41.729901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95015b63-ddb4-48f9-99ac-41d5e3dd601e admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:41.730193 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95015b63-ddb4-48f9-99ac-41d5e3dd601e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:41.730459 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95015b63-ddb4-48f9-99ac-41d5e3dd601e admin admin] PATH_INFO: `/v3/projects/21e72b7ff28248869798ac5cab536382/users/5daec1aa89b5469f99962201673a9173/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:41.752674 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-ba412042-d4d0-4cbe-96bd-3e4a82e02364 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc5801e7e8e7478e9d03165ac63127f5'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:41.758556 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95015b63-ddb4-48f9-99ac-41d5e3dd601e admin admin] RBAC: Authorizing `identity:create_grant(project_id=21e72b7ff28248869798ac5cab536382, user_id=5daec1aa89b5469f99962201673a9173, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:41.761837 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95015b63-ddb4-48f9-99ac-41d5e3dd601e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:41.768900 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-ba412042-d4d0-4cbe-96bd-3e4a82e02364 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc5801e7e8e7478e9d03165ac63127f5'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:41.781714 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-ba412042-d4d0-4cbe-96bd-3e4a82e02364 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc5801e7e8e7478e9d03165ac63127f5'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:41.782579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-ba412042-d4d0-4cbe-96bd-3e4a82e02364 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc5801e7e8e7478e9d03165ac63127f5'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:41.788032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 987/1995] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:41 2022] PUT /identity/v3/projects/21e72b7ff28248869798ac5cab536382/users/5daec1aa89b5469f99962201673a9173/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:41.792887 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95015b63-ddb4-48f9-99ac-41d5e3dd601e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:41.794704 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1009/1996] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:52:41 2022] DELETE /identity/v3/users/dc5801e7e8e7478e9d03165ac63127f5 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:41.797335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-95015b63-ddb4-48f9-99ac-41d5e3dd601e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:41.881857 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95015b63-ddb4-48f9-99ac-41d5e3dd601e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:41.960943 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:41.988556 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553cc141-932c-45e2-b4ce-6c8593aaabef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:41.989421 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553cc141-932c-45e2-b4ce-6c8593aaabef admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:41.989960 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553cc141-932c-45e2-b4ce-6c8593aaabef admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:41.990288 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553cc141-932c-45e2-b4ce-6c8593aaabef admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:41.992395 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553cc141-932c-45e2-b4ce-6c8593aaabef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:41.994396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553cc141-932c-45e2-b4ce-6c8593aaabef admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:42.011016 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 988/1997] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:42.013961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba412042-d4d0-4cbe-96bd-3e4a82e02364 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:42.031482 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba412042-d4d0-4cbe-96bd-3e4a82e02364 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:42.045038 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e4cfc-65ce-42f9-8302-37f393ff017d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:42.046420 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e4cfc-65ce-42f9-8302-37f393ff017d admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:42.046735 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e4cfc-65ce-42f9-8302-37f393ff017d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:42.047096 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e4cfc-65ce-42f9-8302-37f393ff017d admin admin] PATH_INFO: `/v3/projects/21e72b7ff28248869798ac5cab536382/users/5daec1aa89b5469f99962201673a9173/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:42.050368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7e4cfc-65ce-42f9-8302-37f393ff017d admin admin] RBAC: Authorizing `identity:create_grant(project_id=21e72b7ff28248869798ac5cab536382, user_id=5daec1aa89b5469f99962201673a9173, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:42.053822 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7e4cfc-65ce-42f9-8302-37f393ff017d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:42.058278 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:42.077032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1010/1998] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:42 2022] PUT /identity/v3/projects/21e72b7ff28248869798ac5cab536382/users/5daec1aa89b5469f99962201673a9173/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:42.095403 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bee924-8538-4868-91cf-421d9c44b388 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:42.095690 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bee924-8538-4868-91cf-421d9c44b388 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:42.096166 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bee924-8538-4868-91cf-421d9c44b388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:42.183201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-75bee924-8538-4868-91cf-421d9c44b388 None None] MFA Rules not processed for user `5daec1aa89b5469f99962201673a9173`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:42.184225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-75bee924-8538-4868-91cf-421d9c44b388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:42.303900 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 989/1999] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:42 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:42.314208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e4cfc-65ce-42f9-8302-37f393ff017d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:42.317088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-6e7e4cfc-65ce-42f9-8302-37f393ff017d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:42.403934 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e4cfc-65ce-42f9-8302-37f393ff017d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:42.409551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881cef1-be6d-4a59-ad7f-4d3f2f3c1eef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:42.410273 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881cef1-be6d-4a59-ad7f-4d3f2f3c1eef service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:42.410562 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881cef1-be6d-4a59-ad7f-4d3f2f3c1eef service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:42.410836 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881cef1-be6d-4a59-ad7f-4d3f2f3c1eef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:42.422555 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5881cef1-be6d-4a59-ad7f-4d3f2f3c1eef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:42.424100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5881cef1-be6d-4a59-ad7f-4d3f2f3c1eef service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:42.465585 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1011/2000] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:42 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:42.503953 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bee924-8538-4868-91cf-421d9c44b388 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:42.510566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-75bee924-8538-4868-91cf-421d9c44b388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:42.525390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881cef1-be6d-4a59-ad7f-4d3f2f3c1eef service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:42.537703 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881cef1-be6d-4a59-ad7f-4d3f2f3c1eef service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:42.549786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ffe33-3009-4cde-9d4f-18426c22a318 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:42.551227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ffe33-3009-4cde-9d4f-18426c22a318 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:42.552108 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ffe33-3009-4cde-9d4f-18426c22a318 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:42.552108 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ffe33-3009-4cde-9d4f-18426c22a318 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:42.567593 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-de6ffe33-3009-4cde-9d4f-18426c22a318 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:42.663815 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bee924-8538-4868-91cf-421d9c44b388 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:42.675455 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eaa4b7-7f8d-4e93-a074-55d93d31879b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:42.676446 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eaa4b7-7f8d-4e93-a074-55d93d31879b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:42.676828 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eaa4b7-7f8d-4e93-a074-55d93d31879b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:42.677572 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eaa4b7-7f8d-4e93-a074-55d93d31879b admin admin] PATH_INFO: `/v3/projects/ebdb640062a34b7d9b3323b17420d492` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:42.679270 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eaa4b7-7f8d-4e93-a074-55d93d31879b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebdb640062a34b7d9b3323b17420d492)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:42.681630 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eaa4b7-7f8d-4e93-a074-55d93d31879b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:42.690659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ffe33-3009-4cde-9d4f-18426c22a318 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:42.693219 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ffe33-3009-4cde-9d4f-18426c22a318 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:42.714465 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1012/2001] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:52:42.756372 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d9eaa4b7-7f8d-4e93-a074-55d93d31879b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebdb640062a34b7d9b3323b17420d492', 'request_id': 'req-d9eaa4b7-7f8d-4e93-a074-55d93d31879b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:42.766832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d9eaa4b7-7f8d-4e93-a074-55d93d31879b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebdb640062a34b7d9b3323b17420d492', 'request_id': 'req-d9eaa4b7-7f8d-4e93-a074-55d93d31879b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:42.775974 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d9eaa4b7-7f8d-4e93-a074-55d93d31879b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebdb640062a34b7d9b3323b17420d492', 'request_id': 'req-d9eaa4b7-7f8d-4e93-a074-55d93d31879b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:42.794358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d9eaa4b7-7f8d-4e93-a074-55d93d31879b admin admin] The token cache is being invalidate because project ebdb640062a34b7d9b3323b17420d492 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:52:42.794906 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d9eaa4b7-7f8d-4e93-a074-55d93d31879b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:42.796047 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d9eaa4b7-7f8d-4e93-a074-55d93d31879b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:42.798741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 990/2002] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:52:42 2022] DELETE /identity/v3/projects/ebdb640062a34b7d9b3323b17420d492 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:42.804627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af08708-47c4-49f9-9ede-50c455bd7c18 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:42.805088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af08708-47c4-49f9-9ede-50c455bd7c18 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:42.805544 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af08708-47c4-49f9-9ede-50c455bd7c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:42.840141 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-8af08708-47c4-49f9-9ede-50c455bd7c18 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:42.841887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8af08708-47c4-49f9-9ede-50c455bd7c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:42.944230 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1013/2003] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:42.953296 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55d490-0957-40a2-9dff-b26e13da4de1 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:42.953562 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55d490-0957-40a2-9dff-b26e13da4de1 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:42.953819 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55d490-0957-40a2-9dff-b26e13da4de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:42.966945 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:42.997349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ca55d490-0957-40a2-9dff-b26e13da4de1 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:43.001141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ca55d490-0957-40a2-9dff-b26e13da4de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:43.024946 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 991/2004] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:43.030428 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af08708-47c4-49f9-9ede-50c455bd7c18 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:43.046133 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af08708-47c4-49f9-9ede-50c455bd7c18 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:43.057227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6728ea-e465-42f9-b31d-2afb8e4ff466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:43.058406 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6728ea-e465-42f9-b31d-2afb8e4ff466 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:43.058917 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6728ea-e465-42f9-b31d-2afb8e4ff466 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:43.059408 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6728ea-e465-42f9-b31d-2afb8e4ff466 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:43.061050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6728ea-e465-42f9-b31d-2afb8e4ff466 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:52:43.061709 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6728ea-e465-42f9-b31d-2afb8e4ff466 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:43.064619 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6728ea-e465-42f9-b31d-2afb8e4ff466 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:43.086400 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1014/2005] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:52:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:52:43.090587 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55d490-0957-40a2-9dff-b26e13da4de1 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:43.103309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55d490-0957-40a2-9dff-b26e13da4de1 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:43.108950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e72b9e-328d-4633-a2ca-8892f51bbd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:43.109590 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e72b9e-328d-4633-a2ca-8892f51bbd3d admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:43.109874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e72b9e-328d-4633-a2ca-8892f51bbd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:43.110148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e72b9e-328d-4633-a2ca-8892f51bbd3d admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:43.110950 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e72b9e-328d-4633-a2ca-8892f51bbd3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:43.113004 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e72b9e-328d-4633-a2ca-8892f51bbd3d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:43.192561 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 992/2006] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:52:43 2022] POST /identity/v3/projects => generated 369 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:43.196829 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6728ea-e465-42f9-b31d-2afb8e4ff466 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:43.213692 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6728ea-e465-42f9-b31d-2afb8e4ff466 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:43.279248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:43.369964 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7024b-0811-4dc1-ba63-c6537cc154da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:43.371063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7024b-0811-4dc1-ba63-c6537cc154da admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:43.371284 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7024b-0811-4dc1-ba63-c6537cc154da admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:43.371590 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7024b-0811-4dc1-ba63-c6537cc154da admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:43.372607 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7024b-0811-4dc1-ba63-c6537cc154da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:43.375326 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7024b-0811-4dc1-ba63-c6537cc154da admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:43.413839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1015/2007] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:52:43 2022] POST /identity/v3/users => generated 285 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:52:43.414146 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e72b9e-328d-4633-a2ca-8892f51bbd3d admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:43.429480 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e72b9e-328d-4633-a2ca-8892f51bbd3d admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:43.434994 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2480f-d412-4106-a494-eaec942da8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:43.436274 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2480f-d412-4106-a494-eaec942da8b7 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:43.436608 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2480f-d412-4106-a494-eaec942da8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:43.436846 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2480f-d412-4106-a494-eaec942da8b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:43.437663 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2480f-d412-4106-a494-eaec942da8b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:43.439876 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2480f-d412-4106-a494-eaec942da8b7 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:43.457624 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 993/2008] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 08:52:43.462035 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7024b-0811-4dc1-ba63-c6537cc154da admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:43.481729 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7024b-0811-4dc1-ba63-c6537cc154da admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:43.501928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd72138b-8745-4d29-a1b5-ce18827d06e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:43.502582 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72138b-8745-4d29-a1b5-ce18827d06e8 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:43.502859 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72138b-8745-4d29-a1b5-ce18827d06e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:43.503171 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd72138b-8745-4d29-a1b5-ce18827d06e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:43.503842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd72138b-8745-4d29-a1b5-ce18827d06e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:43.505650 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd72138b-8745-4d29-a1b5-ce18827d06e8 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:43.525477 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2480f-d412-4106-a494-eaec942da8b7 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:43.531200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1016/2009] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:43.539785 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2480f-d412-4106-a494-eaec942da8b7 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:43.547354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691775a-0356-4238-b90d-1e6ac3c2e2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:43.548056 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691775a-0356-4238-b90d-1e6ac3c2e2f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:43.548548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691775a-0356-4238-b90d-1e6ac3c2e2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:43.548831 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691775a-0356-4238-b90d-1e6ac3c2e2f0 admin admin] PATH_INFO: `/v3/projects/0785916490864243a6d1e2518481b300/users/164b1e8ed6c44757b5cd9c2d1accd6ab/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:43.573094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691775a-0356-4238-b90d-1e6ac3c2e2f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0785916490864243a6d1e2518481b300, user_id=164b1e8ed6c44757b5cd9c2d1accd6ab, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:43.576430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a691775a-0356-4238-b90d-1e6ac3c2e2f0 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:43.598521 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 994/2010] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:43 2022] PUT /identity/v3/projects/0785916490864243a6d1e2518481b300/users/164b1e8ed6c44757b5cd9c2d1accd6ab/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:43.610731 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87457ce-d9fc-4e80-b866-8a6e48a5dba7 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:43.611181 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87457ce-d9fc-4e80-b866-8a6e48a5dba7 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:43.612012 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87457ce-d9fc-4e80-b866-8a6e48a5dba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:43.646801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-c87457ce-d9fc-4e80-b866-8a6e48a5dba7 None None] MFA Rules not processed for user `164b1e8ed6c44757b5cd9c2d1accd6ab`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:43.648127 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c87457ce-d9fc-4e80-b866-8a6e48a5dba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:43.802552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1017/2011] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:43.815182 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691775a-0356-4238-b90d-1e6ac3c2e2f0 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:43.819263 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a691775a-0356-4238-b90d-1e6ac3c2e2f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:44.005782 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691775a-0356-4238-b90d-1e6ac3c2e2f0 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:44.019653 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fadf46-c5f2-42a3-b83f-1cd88cd67aec service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d178c717a3e49319545e70cb58cbbed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:44.020903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fadf46-c5f2-42a3-b83f-1cd88cd67aec service glance] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:44.021369 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fadf46-c5f2-42a3-b83f-1cd88cd67aec service glance] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:44.021647 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fadf46-c5f2-42a3-b83f-1cd88cd67aec service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:44.037018 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fadf46-c5f2-42a3-b83f-1cd88cd67aec service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:44.039025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fadf46-c5f2-42a3-b83f-1cd88cd67aec service glance] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:44.055711 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 995/2012] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:44.459082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87457ce-d9fc-4e80-b866-8a6e48a5dba7 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:44.461543 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c87457ce-d9fc-4e80-b866-8a6e48a5dba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:44.623945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87457ce-d9fc-4e80-b866-8a6e48a5dba7 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:44.633487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798b56a-1be0-486d-bb1a-75378750f5a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:44.634606 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798b56a-1be0-486d-bb1a-75378750f5a9 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:44.635039 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798b56a-1be0-486d-bb1a-75378750f5a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:44.635324 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798b56a-1be0-486d-bb1a-75378750f5a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:44.638206 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3798b56a-1be0-486d-bb1a-75378750f5a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:44.718473 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:44.749040 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3798b56a-1be0-486d-bb1a-75378750f5a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:44.752545 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3798b56a-1be0-486d-bb1a-75378750f5a9 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:44.764562 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1018/2013] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:44 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:45.162483 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fadf46-c5f2-42a3-b83f-1cd88cd67aec service glance] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:45.164310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-79fadf46-c5f2-42a3-b83f-1cd88cd67aec service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:45.357237 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fadf46-c5f2-42a3-b83f-1cd88cd67aec service glance] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:45.372468 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee5ac9-9318-4c83-9a74-fa5174846ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:45.373404 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee5ac9-9318-4c83-9a74-fa5174846ecf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:45.373759 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee5ac9-9318-4c83-9a74-fa5174846ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:45.374100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee5ac9-9318-4c83-9a74-fa5174846ecf admin admin] PATH_INFO: `/v3/users/fc457fd883664398bdfe033e8fd8c02b` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:45.378345 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee5ac9-9318-4c83-9a74-fa5174846ecf admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc457fd883664398bdfe033e8fd8c02b)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:45.380400 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee5ac9-9318-4c83-9a74-fa5174846ecf admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:45.458109 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:45.512732 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-89ee5ac9-9318-4c83-9a74-fa5174846ecf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc457fd883664398bdfe033e8fd8c02b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:45.533746 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-89ee5ac9-9318-4c83-9a74-fa5174846ecf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc457fd883664398bdfe033e8fd8c02b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:45.558033 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-89ee5ac9-9318-4c83-9a74-fa5174846ecf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc457fd883664398bdfe033e8fd8c02b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:45.561674 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-89ee5ac9-9318-4c83-9a74-fa5174846ecf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc457fd883664398bdfe033e8fd8c02b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:45.562006 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-89ee5ac9-9318-4c83-9a74-fa5174846ecf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc457fd883664398bdfe033e8fd8c02b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:45.665955 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 996/2014] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:52:45 2022] DELETE /identity/v3/users/fc457fd883664398bdfe033e8fd8c02b => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:45.674927 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58480fe9-c351-40b6-8e01-31a842dbd085 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:45.676545 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58480fe9-c351-40b6-8e01-31a842dbd085 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:45.676735 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58480fe9-c351-40b6-8e01-31a842dbd085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:45.720748 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-58480fe9-c351-40b6-8e01-31a842dbd085 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:45.722019 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-58480fe9-c351-40b6-8e01-31a842dbd085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:45.812006 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1019/2015] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:45.823673 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee5ac9-9318-4c83-9a74-fa5174846ecf admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:45.825718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-89ee5ac9-9318-4c83-9a74-fa5174846ecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:45.919948 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee5ac9-9318-4c83-9a74-fa5174846ecf admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:45.924305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d8e7e-f41d-42ad-a219-9146ef952077 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:45.924930 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d8e7e-f41d-42ad-a219-9146ef952077 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:45.925253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d8e7e-f41d-42ad-a219-9146ef952077 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:45.925527 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d8e7e-f41d-42ad-a219-9146ef952077 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:45.946998 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d8e7e-f41d-42ad-a219-9146ef952077 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:45.949025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d8e7e-f41d-42ad-a219-9146ef952077 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:45.973830 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 997/2016] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:52:46.776117 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58480fe9-c351-40b6-8e01-31a842dbd085 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:46.781957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-58480fe9-c351-40b6-8e01-31a842dbd085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:46.807856 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58480fe9-c351-40b6-8e01-31a842dbd085 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:46.824111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881f325-6ea7-42b9-9b73-55de60a26fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:46.826322 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881f325-6ea7-42b9-9b73-55de60a26fc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:46.826770 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881f325-6ea7-42b9-9b73-55de60a26fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:46.827268 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881f325-6ea7-42b9-9b73-55de60a26fc9 admin admin] PATH_INFO: `/v3/projects/cb00ffae95b94eb693c653407294a097` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:46.829484 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881f325-6ea7-42b9-9b73-55de60a26fc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb00ffae95b94eb693c653407294a097)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:46.840344 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881f325-6ea7-42b9-9b73-55de60a26fc9 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:46.913805 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8881f325-6ea7-42b9-9b73-55de60a26fc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb00ffae95b94eb693c653407294a097', 'request_id': 'req-8881f325-6ea7-42b9-9b73-55de60a26fc9'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:46.926482 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8881f325-6ea7-42b9-9b73-55de60a26fc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb00ffae95b94eb693c653407294a097', 'request_id': 'req-8881f325-6ea7-42b9-9b73-55de60a26fc9'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:46.948531 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8881f325-6ea7-42b9-9b73-55de60a26fc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb00ffae95b94eb693c653407294a097', 'request_id': 'req-8881f325-6ea7-42b9-9b73-55de60a26fc9'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:46.964273 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8881f325-6ea7-42b9-9b73-55de60a26fc9 admin admin] The token cache is being invalidate because project cb00ffae95b94eb693c653407294a097 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:52:46.964553 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8881f325-6ea7-42b9-9b73-55de60a26fc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:46.965157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8881f325-6ea7-42b9-9b73-55de60a26fc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:46.966810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1020/2017] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:52:46 2022] DELETE /identity/v3/projects/cb00ffae95b94eb693c653407294a097 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:46.973335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa879a-4360-4ed2-b08b-d3e37c274beb None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:46.973915 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa879a-4360-4ed2-b08b-d3e37c274beb None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:46.973915 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa879a-4360-4ed2-b08b-d3e37c274beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:46.997362 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-3efa879a-4360-4ed2-b08b-d3e37c274beb None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:46.998679 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3efa879a-4360-4ed2-b08b-d3e37c274beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:47.148298 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 998/2018] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:47.156759 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38631666-bb95-499a-b498-ad7aebef2e62 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:47.157044 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38631666-bb95-499a-b498-ad7aebef2e62 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:47.157566 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38631666-bb95-499a-b498-ad7aebef2e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:47.215325 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-38631666-bb95-499a-b498-ad7aebef2e62 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:47.218321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-38631666-bb95-499a-b498-ad7aebef2e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:47.265571 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1021/2019] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:47.269820 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efa879a-4360-4ed2-b08b-d3e37c274beb None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:47.285350 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efa879a-4360-4ed2-b08b-d3e37c274beb None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:47.294170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a86885-d3cd-4b0a-8715-bca2f55f5161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:47.303252 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a86885-d3cd-4b0a-8715-bca2f55f5161 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:47.303543 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a86885-d3cd-4b0a-8715-bca2f55f5161 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:47.306937 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a86885-d3cd-4b0a-8715-bca2f55f5161 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:47.307786 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a86885-d3cd-4b0a-8715-bca2f55f5161 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:52:47.308147 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a86885-d3cd-4b0a-8715-bca2f55f5161 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:47.311114 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a86885-d3cd-4b0a-8715-bca2f55f5161 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:47.331414 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 999/2020] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:52:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:52:47.339478 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38631666-bb95-499a-b498-ad7aebef2e62 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:47.350981 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38631666-bb95-499a-b498-ad7aebef2e62 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:47.356570 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df3d4e-d1f3-4442-bb89-f23515a9b402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:47.357474 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df3d4e-d1f3-4442-bb89-f23515a9b402 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:47.357765 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df3d4e-d1f3-4442-bb89-f23515a9b402 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:47.358129 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df3d4e-d1f3-4442-bb89-f23515a9b402 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:47.358836 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df3d4e-d1f3-4442-bb89-f23515a9b402 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:47.363933 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df3d4e-d1f3-4442-bb89-f23515a9b402 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:47.440298 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1022/2021] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:52:47 2022] POST /identity/v3/projects => generated 371 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:47.445705 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a86885-d3cd-4b0a-8715-bca2f55f5161 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:47.485900 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a86885-d3cd-4b0a-8715-bca2f55f5161 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:47.582201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:47.671193 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86ae3d-821c-438b-8ee9-363a4b7bfff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:47.672373 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86ae3d-821c-438b-8ee9-363a4b7bfff3 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:47.672972 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86ae3d-821c-438b-8ee9-363a4b7bfff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:47.673505 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86ae3d-821c-438b-8ee9-363a4b7bfff3 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:47.674982 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c86ae3d-821c-438b-8ee9-363a4b7bfff3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:47.683155 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c86ae3d-821c-438b-8ee9-363a4b7bfff3 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:47.725188 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1000/2022] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:52:47 2022] POST /identity/v3/users => generated 286 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:47.730300 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df3d4e-d1f3-4442-bb89-f23515a9b402 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:47.774654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df3d4e-d1f3-4442-bb89-f23515a9b402 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:47.791852 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f851dff-62b2-4f0c-a3e2-b8cd29a8742a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:47.792533 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f851dff-62b2-4f0c-a3e2-b8cd29a8742a admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:47.792656 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f851dff-62b2-4f0c-a3e2-b8cd29a8742a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:47.793007 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f851dff-62b2-4f0c-a3e2-b8cd29a8742a admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:47.793829 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f851dff-62b2-4f0c-a3e2-b8cd29a8742a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:47.796311 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f851dff-62b2-4f0c-a3e2-b8cd29a8742a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:47.826923 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1023/2023] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:47.837649 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86ae3d-821c-438b-8ee9-363a4b7bfff3 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:47.879250 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86ae3d-821c-438b-8ee9-363a4b7bfff3 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:47.913375 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab88edc-6d1f-47e2-9174-01c56ddb4095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:47.915087 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab88edc-6d1f-47e2-9174-01c56ddb4095 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:47.915795 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab88edc-6d1f-47e2-9174-01c56ddb4095 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:47.916286 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab88edc-6d1f-47e2-9174-01c56ddb4095 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:47.917771 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab88edc-6d1f-47e2-9174-01c56ddb4095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:47.922549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab88edc-6d1f-47e2-9174-01c56ddb4095 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:47.953488 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1001/2024] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:47.959133 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f851dff-62b2-4f0c-a3e2-b8cd29a8742a admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:47.972367 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f851dff-62b2-4f0c-a3e2-b8cd29a8742a admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:47.980202 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201c7d6-3256-4f11-b513-33085df9c972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:47.981263 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201c7d6-3256-4f11-b513-33085df9c972 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:47.981784 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201c7d6-3256-4f11-b513-33085df9c972 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:47.982369 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201c7d6-3256-4f11-b513-33085df9c972 admin admin] PATH_INFO: `/v3/projects/aeeef876830c4789be1e72353771f258/users/781cb0653d354b1c9577dc68b8153e4b/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:48.002543 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201c7d6-3256-4f11-b513-33085df9c972 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeeef876830c4789be1e72353771f258, user_id=781cb0653d354b1c9577dc68b8153e4b, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:48.006047 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201c7d6-3256-4f11-b513-33085df9c972 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:48.029426 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1024/2025] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:47 2022] PUT /identity/v3/projects/aeeef876830c4789be1e72353771f258/users/781cb0653d354b1c9577dc68b8153e4b/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:48.041505 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b4915-b41a-47b1-b3a2-d00a0a6a7404 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:48.042372 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b4915-b41a-47b1-b3a2-d00a0a6a7404 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:48.042372 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b4915-b41a-47b1-b3a2-d00a0a6a7404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:48.080443 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-3f0b4915-b41a-47b1-b3a2-d00a0a6a7404 None None] MFA Rules not processed for user `781cb0653d354b1c9577dc68b8153e4b`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:48.082040 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3f0b4915-b41a-47b1-b3a2-d00a0a6a7404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:48.222071 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1002/2026] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:48.226963 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201c7d6-3256-4f11-b513-33085df9c972 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:48.235324 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:48.353878 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b4915-b41a-47b1-b3a2-d00a0a6a7404 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:48.355948 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3f0b4915-b41a-47b1-b3a2-d00a0a6a7404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:48.413502 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201c7d6-3256-4f11-b513-33085df9c972 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:48.420263 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f4980-eaf5-4b41-ba40-dc2ecb33aa22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:48.421002 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f4980-eaf5-4b41-ba40-dc2ecb33aa22 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:48.421428 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f4980-eaf5-4b41-ba40-dc2ecb33aa22 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:48.421781 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f4980-eaf5-4b41-ba40-dc2ecb33aa22 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:48.422551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f4980-eaf5-4b41-ba40-dc2ecb33aa22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:48.424693 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f4980-eaf5-4b41-ba40-dc2ecb33aa22 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:48.472519 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1025/2027] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:52:48 2022] POST /identity/v3/projects => generated 375 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:48.479851 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f4980-eaf5-4b41-ba40-dc2ecb33aa22 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:48.495088 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b4915-b41a-47b1-b3a2-d00a0a6a7404 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:48.600005 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:48.669955 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18c1d7-68ea-4204-9d98-21d6f68a11a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:48.671171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18c1d7-68ea-4204-9d98-21d6f68a11a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:48.671856 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18c1d7-68ea-4204-9d98-21d6f68a11a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:48.672349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18c1d7-68ea-4204-9d98-21d6f68a11a8 admin admin] PATH_INFO: `/v3/users/644e9bbd92e14a88b4c5b91712b2e671` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:48.677782 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18c1d7-68ea-4204-9d98-21d6f68a11a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=644e9bbd92e14a88b4c5b91712b2e671)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:48.682592 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18c1d7-68ea-4204-9d98-21d6f68a11a8 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:48.698314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f4980-eaf5-4b41-ba40-dc2ecb33aa22 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:48.712744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac9ed7-79d5-4310-b012-3774f28cce0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:48.712744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac9ed7-79d5-4310-b012-3774f28cce0a admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:48.712744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac9ed7-79d5-4310-b012-3774f28cce0a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:48.713319 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac9ed7-79d5-4310-b012-3774f28cce0a admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:48.713622 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ac9ed7-79d5-4310-b012-3774f28cce0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:48.715887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ac9ed7-79d5-4310-b012-3774f28cce0a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:48.759368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1026/2028] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:52:48 2022] POST /identity/v3/users => generated 294 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:52:48.764762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac9ed7-79d5-4310-b012-3774f28cce0a admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:48.770428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fc18c1d7-68ea-4204-9d98-21d6f68a11a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '644e9bbd92e14a88b4c5b91712b2e671'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:48.779918 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac9ed7-79d5-4310-b012-3774f28cce0a admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:48.788434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab71978-3207-4b0a-a9c2-16cccd332213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:48.789543 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab71978-3207-4b0a-a9c2-16cccd332213 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:48.790047 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab71978-3207-4b0a-a9c2-16cccd332213 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:48.790396 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab71978-3207-4b0a-a9c2-16cccd332213 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:48.791781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fc18c1d7-68ea-4204-9d98-21d6f68a11a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '644e9bbd92e14a88b4c5b91712b2e671'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:48.792543 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab71978-3207-4b0a-a9c2-16cccd332213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:48.797752 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab71978-3207-4b0a-a9c2-16cccd332213 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:48.805241 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fc18c1d7-68ea-4204-9d98-21d6f68a11a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '644e9bbd92e14a88b4c5b91712b2e671'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:48.806521 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fc18c1d7-68ea-4204-9d98-21d6f68a11a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '644e9bbd92e14a88b4c5b91712b2e671'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:48.807922 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-fc18c1d7-68ea-4204-9d98-21d6f68a11a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '644e9bbd92e14a88b4c5b91712b2e671'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:48.828028 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1027/2029] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:48 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:48.834619 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab71978-3207-4b0a-a9c2-16cccd332213 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:48.839008 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4ab71978-3207-4b0a-a9c2-16cccd332213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:48.851310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1003/2030] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:52:48 2022] DELETE /identity/v3/users/644e9bbd92e14a88b4c5b91712b2e671 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:52:48.860160 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc86d0-1702-445e-a31d-39540c018d27 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:48.860650 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc86d0-1702-445e-a31d-39540c018d27 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:48.861051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc86d0-1702-445e-a31d-39540c018d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:48.873289 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab71978-3207-4b0a-a9c2-16cccd332213 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:48.891171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-4bdc86d0-1702-445e-a31d-39540c018d27 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:48.893023 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4bdc86d0-1702-445e-a31d-39540c018d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:48.971668 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27000622-a323-464b-ac23-83a7a3a2214b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:48.972351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27000622-a323-464b-ac23-83a7a3a2214b admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:48.972654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27000622-a323-464b-ac23-83a7a3a2214b admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:48.972947 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27000622-a323-464b-ac23-83a7a3a2214b admin admin] PATH_INFO: `/v3/projects/d81c1b6bde7b46e8ab03ea764e160655/users/e5058323532245d5afb4b8f3c8f176bd/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:48.986672 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:48.989434 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1004/2031] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:49.005330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc86d0-1702-445e-a31d-39540c018d27 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:49.007956 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27000622-a323-464b-ac23-83a7a3a2214b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d81c1b6bde7b46e8ab03ea764e160655, user_id=e5058323532245d5afb4b8f3c8f176bd, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:49.008724 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4bdc86d0-1702-445e-a31d-39540c018d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:49.014625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27000622-a323-464b-ac23-83a7a3a2214b admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:49.045236 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1028/2032] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:48 2022] PUT /identity/v3/projects/d81c1b6bde7b46e8ab03ea764e160655/users/e5058323532245d5afb4b8f3c8f176bd/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:49.049769 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27000622-a323-464b-ac23-83a7a3a2214b admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:49.112319 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc86d0-1702-445e-a31d-39540c018d27 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:49.144953 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:49.149699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27000622-a323-464b-ac23-83a7a3a2214b admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:49.161406 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9497245-0e2d-4ef3-b014-00b26ceab1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:49.162678 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9497245-0e2d-4ef3-b014-00b26ceab1a8 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:49.162678 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9497245-0e2d-4ef3-b014-00b26ceab1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:49.162678 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9497245-0e2d-4ef3-b014-00b26ceab1a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:49.163863 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9497245-0e2d-4ef3-b014-00b26ceab1a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:49.166330 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9497245-0e2d-4ef3-b014-00b26ceab1a8 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:49.185869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1029/2033] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 08:52:49.193374 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9497245-0e2d-4ef3-b014-00b26ceab1a8 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:49.208594 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9497245-0e2d-4ef3-b014-00b26ceab1a8 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:49.217440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046ff44-b7c7-434d-88d4-bd9421700202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:49.218739 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046ff44-b7c7-434d-88d4-bd9421700202 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:49.220160 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212598c2-4c11-4e94-8c82-540a62e1399e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:49.220646 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046ff44-b7c7-434d-88d4-bd9421700202 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:49.220921 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046ff44-b7c7-434d-88d4-bd9421700202 admin admin] PATH_INFO: `/v3/projects/d81c1b6bde7b46e8ab03ea764e160655/users/e5058323532245d5afb4b8f3c8f176bd/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:49.222246 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212598c2-4c11-4e94-8c82-540a62e1399e service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:49.222738 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212598c2-4c11-4e94-8c82-540a62e1399e service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:49.223174 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212598c2-4c11-4e94-8c82-540a62e1399e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:49.223897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046ff44-b7c7-434d-88d4-bd9421700202 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d81c1b6bde7b46e8ab03ea764e160655, user_id=e5058323532245d5afb4b8f3c8f176bd, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:49.228083 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046ff44-b7c7-434d-88d4-bd9421700202 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:49.245180 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1030/2034] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:49 2022] PUT /identity/v3/projects/d81c1b6bde7b46e8ab03ea764e160655/users/e5058323532245d5afb4b8f3c8f176bd/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:52:49.246926 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212598c2-4c11-4e94-8c82-540a62e1399e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:49.249931 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212598c2-4c11-4e94-8c82-540a62e1399e service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:49.256165 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bbcce-2923-4a64-9c70-d1d6c2eb1f00 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:49.256461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bbcce-2923-4a64-9c70-d1d6c2eb1f00 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:49.256840 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bbcce-2923-4a64-9c70-d1d6c2eb1f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:49.271940 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1005/2035] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:49.325055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-697bbcce-2923-4a64-9c70-d1d6c2eb1f00 None None] MFA Rules not processed for user `e5058323532245d5afb4b8f3c8f176bd`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:49.327552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-697bbcce-2923-4a64-9c70-d1d6c2eb1f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:49.483698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1031/2036] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:52:49.500563 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212598c2-4c11-4e94-8c82-540a62e1399e service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:49.502674 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-212598c2-4c11-4e94-8c82-540a62e1399e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:49.621476 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212598c2-4c11-4e94-8c82-540a62e1399e service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:49.727051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299bb96-77df-4483-8560-6f2d276a4128 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:49.727538 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299bb96-77df-4483-8560-6f2d276a4128 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:49.727800 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299bb96-77df-4483-8560-6f2d276a4128 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:49.728072 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299bb96-77df-4483-8560-6f2d276a4128 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:49.730336 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5299bb96-77df-4483-8560-6f2d276a4128 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:49.845635 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5299bb96-77df-4483-8560-6f2d276a4128 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:49.847152 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5299bb96-77df-4483-8560-6f2d276a4128 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:49.863401 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1006/2037] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:49.980104 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697bbcce-2923-4a64-9c70-d1d6c2eb1f00 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:49.983352 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-697bbcce-2923-4a64-9c70-d1d6c2eb1f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:50.100689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:50.117403 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697bbcce-2923-4a64-9c70-d1d6c2eb1f00 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:50.127377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6494fa-c115-4b55-a261-b9bc95fbaaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:50.128619 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6494fa-c115-4b55-a261-b9bc95fbaaf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:50.129170 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6494fa-c115-4b55-a261-b9bc95fbaaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:50.129745 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6494fa-c115-4b55-a261-b9bc95fbaaf7 admin admin] PATH_INFO: `/v3/projects/9b008635fdf24cc7b5cceb0ceab1e75c` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:50.132858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6494fa-c115-4b55-a261-b9bc95fbaaf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b008635fdf24cc7b5cceb0ceab1e75c)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:50.136868 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6494fa-c115-4b55-a261-b9bc95fbaaf7 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:50.235515 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cd6494fa-c115-4b55-a261-b9bc95fbaaf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b008635fdf24cc7b5cceb0ceab1e75c', 'request_id': 'req-cd6494fa-c115-4b55-a261-b9bc95fbaaf7'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:50.255086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cd6494fa-c115-4b55-a261-b9bc95fbaaf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b008635fdf24cc7b5cceb0ceab1e75c', 'request_id': 'req-cd6494fa-c115-4b55-a261-b9bc95fbaaf7'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:50.271462 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cd6494fa-c115-4b55-a261-b9bc95fbaaf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b008635fdf24cc7b5cceb0ceab1e75c', 'request_id': 'req-cd6494fa-c115-4b55-a261-b9bc95fbaaf7'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:50.312422 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cd6494fa-c115-4b55-a261-b9bc95fbaaf7 admin admin] The token cache is being invalidate because project 9b008635fdf24cc7b5cceb0ceab1e75c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:52:50.312856 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cd6494fa-c115-4b55-a261-b9bc95fbaaf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:50.313640 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cd6494fa-c115-4b55-a261-b9bc95fbaaf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:50.316318 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1032/2038] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:52:49 2022] DELETE /identity/v3/projects/9b008635fdf24cc7b5cceb0ceab1e75c => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:50.321589 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299bb96-77df-4483-8560-6f2d276a4128 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:50.338879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299bb96-77df-4483-8560-6f2d276a4128 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:50.421741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:50.489140 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2a364-31a4-4495-9b71-2eb3038b625c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:50.489901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2a364-31a4-4495-9b71-2eb3038b625c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:50.490360 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2a364-31a4-4495-9b71-2eb3038b625c admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:50.491435 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2a364-31a4-4495-9b71-2eb3038b625c admin admin] PATH_INFO: `/v3/users/5daec1aa89b5469f99962201673a9173` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:50.494446 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2a364-31a4-4495-9b71-2eb3038b625c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5daec1aa89b5469f99962201673a9173)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:50.497320 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2a364-31a4-4495-9b71-2eb3038b625c admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:50.609178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-41d2a364-31a4-4495-9b71-2eb3038b625c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5daec1aa89b5469f99962201673a9173'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:50.622120 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6494fa-c115-4b55-a261-b9bc95fbaaf7 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:50.625131 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-cd6494fa-c115-4b55-a261-b9bc95fbaaf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:50.627204 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-41d2a364-31a4-4495-9b71-2eb3038b625c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5daec1aa89b5469f99962201673a9173'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:50.647723 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-41d2a364-31a4-4495-9b71-2eb3038b625c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5daec1aa89b5469f99962201673a9173'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:50.649281 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-41d2a364-31a4-4495-9b71-2eb3038b625c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5daec1aa89b5469f99962201673a9173'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:50.649500 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6494fa-c115-4b55-a261-b9bc95fbaaf7 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:50.651938 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-41d2a364-31a4-4495-9b71-2eb3038b625c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5daec1aa89b5469f99962201673a9173'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:50.663304 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8bd6a5-ade6-4a99-867b-24f2a61314ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:50.666150 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8bd6a5-ade6-4a99-867b-24f2a61314ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:50.669517 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8bd6a5-ade6-4a99-867b-24f2a61314ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:50.671473 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8bd6a5-ade6-4a99-867b-24f2a61314ef admin admin] PATH_INFO: `/v3/users/4f9a447a8d46446fa1eedda883212be9` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:50.674534 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8bd6a5-ade6-4a99-867b-24f2a61314ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f9a447a8d46446fa1eedda883212be9)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:50.678941 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8bd6a5-ade6-4a99-867b-24f2a61314ef admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:50.707549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1007/2039] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:52:50 2022] DELETE /identity/v3/users/5daec1aa89b5469f99962201673a9173 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:50.785614 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5d8bd6a5-ade6-4a99-867b-24f2a61314ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f9a447a8d46446fa1eedda883212be9'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:50.818145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5d8bd6a5-ade6-4a99-867b-24f2a61314ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f9a447a8d46446fa1eedda883212be9'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:50.846068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5d8bd6a5-ade6-4a99-867b-24f2a61314ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f9a447a8d46446fa1eedda883212be9'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:50.871021 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5d8bd6a5-ade6-4a99-867b-24f2a61314ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f9a447a8d46446fa1eedda883212be9'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:50.872200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5d8bd6a5-ade6-4a99-867b-24f2a61314ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f9a447a8d46446fa1eedda883212be9'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:50.890807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1033/2040] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:52:50 2022] DELETE /identity/v3/users/4f9a447a8d46446fa1eedda883212be9 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:50.897277 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69621584-52fc-4176-a0a8-afbbd7d9dc82 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:50.897847 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69621584-52fc-4176-a0a8-afbbd7d9dc82 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:50.898330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69621584-52fc-4176-a0a8-afbbd7d9dc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:50.947861 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-69621584-52fc-4176-a0a8-afbbd7d9dc82 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:50.950019 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-69621584-52fc-4176-a0a8-afbbd7d9dc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:51.108126 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1008/2041] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:51.119066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8bd6a5-ade6-4a99-867b-24f2a61314ef admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:51.120648 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-5d8bd6a5-ade6-4a99-867b-24f2a61314ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:51.204014 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8bd6a5-ade6-4a99-867b-24f2a61314ef admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:51.208641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d74266-5bb4-4341-9be7-e0556b398fbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:51.209293 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d74266-5bb4-4341-9be7-e0556b398fbd service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:51.209639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d74266-5bb4-4341-9be7-e0556b398fbd service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:51.209931 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d74266-5bb4-4341-9be7-e0556b398fbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:51.222019 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d74266-5bb4-4341-9be7-e0556b398fbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:51.222019 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d74266-5bb4-4341-9be7-e0556b398fbd service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:51.232648 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1034/2042] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:51.692694 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69621584-52fc-4176-a0a8-afbbd7d9dc82 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:51.695076 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-69621584-52fc-4176-a0a8-afbbd7d9dc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:51.703572 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d74266-5bb4-4341-9be7-e0556b398fbd service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:51.705164 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c7d74266-5bb4-4341-9be7-e0556b398fbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:51.734328 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d74266-5bb4-4341-9be7-e0556b398fbd service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:51.736795 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:51.740917 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9e6b5-fea3-4f9c-a34a-61b022fd2d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:51.742813 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69621584-52fc-4176-a0a8-afbbd7d9dc82 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:51.743026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9e6b5-fea3-4f9c-a34a-61b022fd2d53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:51.743275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9e6b5-fea3-4f9c-a34a-61b022fd2d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:51.744038 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9e6b5-fea3-4f9c-a34a-61b022fd2d53 admin admin] PATH_INFO: `/v3/projects/21e72b7ff28248869798ac5cab536382` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:51.746331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9e6b5-fea3-4f9c-a34a-61b022fd2d53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21e72b7ff28248869798ac5cab536382)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:51.749198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9e6b5-fea3-4f9c-a34a-61b022fd2d53 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:51.751967 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1655d-23ef-4fd7-be4f-c54b4248cfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:51.752738 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1655d-23ef-4fd7-be4f-c54b4248cfa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:51.753091 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1655d-23ef-4fd7-be4f-c54b4248cfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:51.753353 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1655d-23ef-4fd7-be4f-c54b4248cfa5 admin admin] PATH_INFO: `/v3/projects/1a93639f05644b30aa8ff9e9c2078126` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:51.754864 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1655d-23ef-4fd7-be4f-c54b4248cfa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a93639f05644b30aa8ff9e9c2078126)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:51.757062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1655d-23ef-4fd7-be4f-c54b4248cfa5 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:51.851717 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-77d9e6b5-fea3-4f9c-a34a-61b022fd2d53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21e72b7ff28248869798ac5cab536382', 'request_id': 'req-77d9e6b5-fea3-4f9c-a34a-61b022fd2d53'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:51.857670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-dee1655d-23ef-4fd7-be4f-c54b4248cfa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a93639f05644b30aa8ff9e9c2078126', 'request_id': 'req-dee1655d-23ef-4fd7-be4f-c54b4248cfa5'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:51.858187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-77d9e6b5-fea3-4f9c-a34a-61b022fd2d53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21e72b7ff28248869798ac5cab536382', 'request_id': 'req-77d9e6b5-fea3-4f9c-a34a-61b022fd2d53'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:51.864065 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:51.877834 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-77d9e6b5-fea3-4f9c-a34a-61b022fd2d53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21e72b7ff28248869798ac5cab536382', 'request_id': 'req-77d9e6b5-fea3-4f9c-a34a-61b022fd2d53'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:51.878505 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-dee1655d-23ef-4fd7-be4f-c54b4248cfa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a93639f05644b30aa8ff9e9c2078126', 'request_id': 'req-dee1655d-23ef-4fd7-be4f-c54b4248cfa5'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:51.897290 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-dee1655d-23ef-4fd7-be4f-c54b4248cfa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a93639f05644b30aa8ff9e9c2078126', 'request_id': 'req-dee1655d-23ef-4fd7-be4f-c54b4248cfa5'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:51.903293 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-77d9e6b5-fea3-4f9c-a34a-61b022fd2d53 admin admin] The token cache is being invalidate because project 21e72b7ff28248869798ac5cab536382 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:52:51.905323 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-77d9e6b5-fea3-4f9c-a34a-61b022fd2d53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:51.907303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-77d9e6b5-fea3-4f9c-a34a-61b022fd2d53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:51.917462 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1035/2043] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:52:51 2022] DELETE /identity/v3/projects/21e72b7ff28248869798ac5cab536382 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:52:51.918229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d625676-963d-4ced-8c1d-e1d00a03cd13 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:51.918512 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d625676-963d-4ced-8c1d-e1d00a03cd13 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:51.918791 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d625676-963d-4ced-8c1d-e1d00a03cd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:51.927216 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-dee1655d-23ef-4fd7-be4f-c54b4248cfa5 admin admin] The token cache is being invalidate because project 1a93639f05644b30aa8ff9e9c2078126 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:52:51.927679 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-dee1655d-23ef-4fd7-be4f-c54b4248cfa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:51.929112 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-dee1655d-23ef-4fd7-be4f-c54b4248cfa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:51.933476 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1009/2044] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:52:51 2022] DELETE /identity/v3/projects/1a93639f05644b30aa8ff9e9c2078126 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:51.939626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1655d-23ef-4fd7-be4f-c54b4248cfa5 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:51.944810 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-dee1655d-23ef-4fd7-be4f-c54b4248cfa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:52.007652 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-3d625676-963d-4ced-8c1d-e1d00a03cd13 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:52.009170 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3d625676-963d-4ced-8c1d-e1d00a03cd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:52.090284 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1655d-23ef-4fd7-be4f-c54b4248cfa5 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:52.119720 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1036/2045] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:52:52.128697 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5670e-5bc5-4c76-a2a1-32ac163a1582 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:52.128997 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5670e-5bc5-4c76-a2a1-32ac163a1582 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:52.129300 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5670e-5bc5-4c76-a2a1-32ac163a1582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:52.151707 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-60c5670e-5bc5-4c76-a2a1-32ac163a1582 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:52.153145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-60c5670e-5bc5-4c76-a2a1-32ac163a1582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:52.162422 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1037/2046] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:52:52.166972 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5670e-5bc5-4c76-a2a1-32ac163a1582 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:52.177473 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5670e-5bc5-4c76-a2a1-32ac163a1582 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:52.183003 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2826f-cc9a-4d98-a09b-ce03a9125ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:52.183636 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2826f-cc9a-4d98-a09b-ce03a9125ace admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:52.183912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2826f-cc9a-4d98-a09b-ce03a9125ace admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:52.184182 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2826f-cc9a-4d98-a09b-ce03a9125ace admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:52.185199 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b2826f-cc9a-4d98-a09b-ce03a9125ace admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:52:52.185199 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b2826f-cc9a-4d98-a09b-ce03a9125ace admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:52.187289 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b2826f-cc9a-4d98-a09b-ce03a9125ace admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:52.207790 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1038/2047] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:52:52.212339 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3971f0-8471-4485-8101-964e94d04440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:52.213459 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3971f0-8471-4485-8101-964e94d04440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:52.213931 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2826f-cc9a-4d98-a09b-ce03a9125ace admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:52.214078 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3971f0-8471-4485-8101-964e94d04440 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:52.214396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3971f0-8471-4485-8101-964e94d04440 admin admin] PATH_INFO: `/v3/users/2c72c60887114c2eb6eff02d223f6355` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:52.217502 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3971f0-8471-4485-8101-964e94d04440 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c72c60887114c2eb6eff02d223f6355)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:52.220651 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3971f0-8471-4485-8101-964e94d04440 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:52.233633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2826f-cc9a-4d98-a09b-ce03a9125ace admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:52.242938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d33a80-3dab-43c9-99a9-ea6aa2c67b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:52.243586 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d33a80-3dab-43c9-99a9-ea6aa2c67b65 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:52.243862 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d33a80-3dab-43c9-99a9-ea6aa2c67b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:52.244130 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d33a80-3dab-43c9-99a9-ea6aa2c67b65 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:52.244768 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d33a80-3dab-43c9-99a9-ea6aa2c67b65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:52.246733 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d33a80-3dab-43c9-99a9-ea6aa2c67b65 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:52.282947 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1039/2048] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:52:52 2022] POST /identity/v3/projects => generated 405 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:52.288419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d33a80-3dab-43c9-99a9-ea6aa2c67b65 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:52.310150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3b3971f0-8471-4485-8101-964e94d04440 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c72c60887114c2eb6eff02d223f6355'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:52.325274 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3b3971f0-8471-4485-8101-964e94d04440 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c72c60887114c2eb6eff02d223f6355'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:52.333892 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3b3971f0-8471-4485-8101-964e94d04440 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c72c60887114c2eb6eff02d223f6355'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:52.334510 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3b3971f0-8471-4485-8101-964e94d04440 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c72c60887114c2eb6eff02d223f6355'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:52.335435 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3b3971f0-8471-4485-8101-964e94d04440 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c72c60887114c2eb6eff02d223f6355'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:52.353055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:52.360981 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1010/2049] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:52:51 2022] DELETE /identity/v3/users/2c72c60887114c2eb6eff02d223f6355 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:52.385065 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d33a80-3dab-43c9-99a9-ea6aa2c67b65 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:52.453642 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc828c2-720b-4b22-bf62-98128bbb36ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:52.454667 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc828c2-720b-4b22-bf62-98128bbb36ca admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:52.454981 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc828c2-720b-4b22-bf62-98128bbb36ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:52.455254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc828c2-720b-4b22-bf62-98128bbb36ca admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:52.455949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc828c2-720b-4b22-bf62-98128bbb36ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:52.457885 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc828c2-720b-4b22-bf62-98128bbb36ca admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:52.481725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1040/2050] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:52:52 2022] POST /identity/v3/users => generated 303 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:52.486114 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3971f0-8471-4485-8101-964e94d04440 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:52.487603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3b3971f0-8471-4485-8101-964e94d04440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:52.502652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3971f0-8471-4485-8101-964e94d04440 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:52.508145 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb429f9-3a3b-457a-9d62-823d82de915f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:52.508931 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb429f9-3a3b-457a-9d62-823d82de915f admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:52.509211 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb429f9-3a3b-457a-9d62-823d82de915f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:52.509483 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb429f9-3a3b-457a-9d62-823d82de915f admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:52.510160 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb429f9-3a3b-457a-9d62-823d82de915f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:52.511863 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb429f9-3a3b-457a-9d62-823d82de915f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:52.513537 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:52.545783 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1011/2051] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:52.549349 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc828c2-720b-4b22-bf62-98128bbb36ca admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:52.562863 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc828c2-720b-4b22-bf62-98128bbb36ca admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:52.571952 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8565b2d9-23f0-443f-8d55-b2a6d1a1a05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:52.572878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565b2d9-23f0-443f-8d55-b2a6d1a1a05b admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:52.573294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565b2d9-23f0-443f-8d55-b2a6d1a1a05b admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:52.573712 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8565b2d9-23f0-443f-8d55-b2a6d1a1a05b admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:52.574772 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8565b2d9-23f0-443f-8d55-b2a6d1a1a05b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:52.579153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8565b2d9-23f0-443f-8d55-b2a6d1a1a05b admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:52.603769 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1041/2052] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:52.610518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb429f9-3a3b-457a-9d62-823d82de915f admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:52.629181 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb429f9-3a3b-457a-9d62-823d82de915f admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:52.637309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28cb165-9cd9-4efd-9970-9b18eccb36c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:52.638837 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cb165-9cd9-4efd-9970-9b18eccb36c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:52.639414 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cb165-9cd9-4efd-9970-9b18eccb36c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:52.640111 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cb165-9cd9-4efd-9970-9b18eccb36c5 admin admin] PATH_INFO: `/v3/projects/d013d9db6425442293db93fb16b172cf/users/4f66ad0b219843ceb7c33aa9cbc58a34/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:52.667453 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28cb165-9cd9-4efd-9970-9b18eccb36c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d013d9db6425442293db93fb16b172cf, user_id=4f66ad0b219843ceb7c33aa9cbc58a34, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:52.670251 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28cb165-9cd9-4efd-9970-9b18eccb36c5 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:52.693237 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1012/2053] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:52 2022] PUT /identity/v3/projects/d013d9db6425442293db93fb16b172cf/users/4f66ad0b219843ceb7c33aa9cbc58a34/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:52.707243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15204b-2ef7-49c8-8815-3dcbf309a110 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:52.707585 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15204b-2ef7-49c8-8815-3dcbf309a110 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:52.707920 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15204b-2ef7-49c8-8815-3dcbf309a110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:52.717938 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28cb165-9cd9-4efd-9970-9b18eccb36c5 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:52.720372 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d28cb165-9cd9-4efd-9970-9b18eccb36c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:52.764984 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ab15204b-2ef7-49c8-8815-3dcbf309a110 None None] MFA Rules not processed for user `4f66ad0b219843ceb7c33aa9cbc58a34`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:52.766256 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ab15204b-2ef7-49c8-8815-3dcbf309a110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:52.798943 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28cb165-9cd9-4efd-9970-9b18eccb36c5 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:52.803740 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f7dad-8715-4b93-bd81-6e0cf7674401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:52.804539 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f7dad-8715-4b93-bd81-6e0cf7674401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:52.804841 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f7dad-8715-4b93-bd81-6e0cf7674401 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:52.805113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f7dad-8715-4b93-bd81-6e0cf7674401 admin admin] PATH_INFO: `/v3/projects/bbef6d692ab04ff4998d5f1564e3634f` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:52.806265 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f7dad-8715-4b93-bd81-6e0cf7674401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbef6d692ab04ff4998d5f1564e3634f)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:52.808297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f7dad-8715-4b93-bd81-6e0cf7674401 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:52.847551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1042/2054] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:52 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:52:52.851654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15204b-2ef7-49c8-8815-3dcbf309a110 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:52.851654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ab15204b-2ef7-49c8-8815-3dcbf309a110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:52.864919 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15204b-2ef7-49c8-8815-3dcbf309a110 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:52.867134 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4c4f7dad-8715-4b93-bd81-6e0cf7674401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbef6d692ab04ff4998d5f1564e3634f', 'request_id': 'req-4c4f7dad-8715-4b93-bd81-6e0cf7674401'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:52.877124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4c4f7dad-8715-4b93-bd81-6e0cf7674401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbef6d692ab04ff4998d5f1564e3634f', 'request_id': 'req-4c4f7dad-8715-4b93-bd81-6e0cf7674401'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:52.883998 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4c4f7dad-8715-4b93-bd81-6e0cf7674401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbef6d692ab04ff4998d5f1564e3634f', 'request_id': 'req-4c4f7dad-8715-4b93-bd81-6e0cf7674401'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:52.895491 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4c4f7dad-8715-4b93-bd81-6e0cf7674401 admin admin] The token cache is being invalidate because project bbef6d692ab04ff4998d5f1564e3634f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:52:52.895779 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4c4f7dad-8715-4b93-bd81-6e0cf7674401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:52.896538 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4c4f7dad-8715-4b93-bd81-6e0cf7674401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:52.898847 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1013/2055] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:52:52 2022] DELETE /identity/v3/projects/bbef6d692ab04ff4998d5f1564e3634f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:52.900013 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f7dad-8715-4b93-bd81-6e0cf7674401 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:52.901570 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4c4f7dad-8715-4b93-bd81-6e0cf7674401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:52.930609 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8d3df-ba21-42aa-98f7-405f96f2a205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:52.931455 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8d3df-ba21-42aa-98f7-405f96f2a205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:52.931738 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8d3df-ba21-42aa-98f7-405f96f2a205 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:52.932006 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8d3df-ba21-42aa-98f7-405f96f2a205 admin admin] PATH_INFO: `/v3/users/781cb0653d354b1c9577dc68b8153e4b` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:52.933250 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8d3df-ba21-42aa-98f7-405f96f2a205 admin admin] RBAC: Authorizing `identity:delete_user(user_id=781cb0653d354b1c9577dc68b8153e4b)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:52.935391 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8d3df-ba21-42aa-98f7-405f96f2a205 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:52.938287 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:52.993063 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f7dad-8715-4b93-bd81-6e0cf7674401 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:52.999357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c03ba8e-7811-4705-9a31-7004f599dbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:53.000126 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03ba8e-7811-4705-9a31-7004f599dbee admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:53.000329 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03ba8e-7811-4705-9a31-7004f599dbee admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:53.000534 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03ba8e-7811-4705-9a31-7004f599dbee admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:53.001199 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c03ba8e-7811-4705-9a31-7004f599dbee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:53.003377 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c03ba8e-7811-4705-9a31-7004f599dbee admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:53.010242 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:53.017465 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-16e8d3df-ba21-42aa-98f7-405f96f2a205 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '781cb0653d354b1c9577dc68b8153e4b'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:53.033934 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-16e8d3df-ba21-42aa-98f7-405f96f2a205 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '781cb0653d354b1c9577dc68b8153e4b'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:53.044249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-16e8d3df-ba21-42aa-98f7-405f96f2a205 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '781cb0653d354b1c9577dc68b8153e4b'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:53.058104 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1014/2056] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:52:52 2022] POST /identity/v3/projects => generated 409 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:53.058104 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba64a21-cbfa-4286-b746-3cbbb8ec11c3 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:53.058104 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba64a21-cbfa-4286-b746-3cbbb8ec11c3 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:53.058104 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba64a21-cbfa-4286-b746-3cbbb8ec11c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:53.058488 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-16e8d3df-ba21-42aa-98f7-405f96f2a205 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '781cb0653d354b1c9577dc68b8153e4b'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:53.059359 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-16e8d3df-ba21-42aa-98f7-405f96f2a205 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '781cb0653d354b1c9577dc68b8153e4b'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:53.068879 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1043/2057] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:52:52 2022] DELETE /identity/v3/users/781cb0653d354b1c9577dc68b8153e4b => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:53.074061 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8d3df-ba21-42aa-98f7-405f96f2a205 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:53.074061 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-16e8d3df-ba21-42aa-98f7-405f96f2a205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:53.089831 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-7ba64a21-cbfa-4286-b746-3cbbb8ec11c3 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:53.091246 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7ba64a21-cbfa-4286-b746-3cbbb8ec11c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:53.159085 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8d3df-ba21-42aa-98f7-405f96f2a205 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:53.164413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456a0f9f-726f-4f62-987f-6e50e47a3ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:53.165082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a0f9f-726f-4f62-987f-6e50e47a3ef3 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:53.165448 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a0f9f-726f-4f62-987f-6e50e47a3ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:53.165794 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a0f9f-726f-4f62-987f-6e50e47a3ef3 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:53.166528 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456a0f9f-726f-4f62-987f-6e50e47a3ef3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:53.168588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456a0f9f-726f-4f62-987f-6e50e47a3ef3 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:53.192903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1015/2058] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:53.194810 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d52c0-15a5-46a6-ba13-205d72c29107 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:53.195137 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d52c0-15a5-46a6-ba13-205d72c29107 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:53.195408 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d52c0-15a5-46a6-ba13-205d72c29107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:53.206984 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1044/2059] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:52:53 2022] POST /identity/v3/users => generated 311 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:53.211977 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87677038-7c47-412e-9294-bb8a535ce9b8 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:53.212347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87677038-7c47-412e-9294-bb8a535ce9b8 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:53.212700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87677038-7c47-412e-9294-bb8a535ce9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:53.219405 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-d10d52c0-15a5-46a6-ba13-205d72c29107 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:53.220639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d10d52c0-15a5-46a6-ba13-205d72c29107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:53.234054 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1016/2060] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:53.234054 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d52c0-15a5-46a6-ba13-205d72c29107 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:53.241582 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-87677038-7c47-412e-9294-bb8a535ce9b8 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:53.241782 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d52c0-15a5-46a6-ba13-205d72c29107 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:53.243369 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-87677038-7c47-412e-9294-bb8a535ce9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:53.246112 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6ba82-6ce2-47a3-a5c4-1ead644549f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:53.246734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6ba82-6ce2-47a3-a5c4-1ead644549f1 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:53.247048 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6ba82-6ce2-47a3-a5c4-1ead644549f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:53.247304 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6ba82-6ce2-47a3-a5c4-1ead644549f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:53.248045 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6ba82-6ce2-47a3-a5c4-1ead644549f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:53.249864 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6ba82-6ce2-47a3-a5c4-1ead644549f1 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:53.254820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1045/2061] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:53.256666 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87677038-7c47-412e-9294-bb8a535ce9b8 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:53.258431 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-87677038-7c47-412e-9294-bb8a535ce9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:53.261253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1017/2062] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:53.261969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6ba82-6ce2-47a3-a5c4-1ead644549f1 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:53.277209 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6ba82-6ce2-47a3-a5c4-1ead644549f1 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:53.282144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8cdec8-af81-43cd-893e-0a1cb5f1fafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:53.282792 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8cdec8-af81-43cd-893e-0a1cb5f1fafa admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:53.283128 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8cdec8-af81-43cd-893e-0a1cb5f1fafa admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:53.283410 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8cdec8-af81-43cd-893e-0a1cb5f1fafa admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:53.284089 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8cdec8-af81-43cd-893e-0a1cb5f1fafa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:52:53.284836 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8cdec8-af81-43cd-893e-0a1cb5f1fafa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:53.286063 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8cdec8-af81-43cd-893e-0a1cb5f1fafa admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:53.303579 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1018/2063] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:52:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:52:53.309213 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8cdec8-af81-43cd-893e-0a1cb5f1fafa admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:53.320920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8cdec8-af81-43cd-893e-0a1cb5f1fafa admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:53.325767 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34321d-f1e1-4d61-ba56-921ec56ded3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:53.326492 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34321d-f1e1-4d61-ba56-921ec56ded3f admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:53.326828 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34321d-f1e1-4d61-ba56-921ec56ded3f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:53.327076 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34321d-f1e1-4d61-ba56-921ec56ded3f admin admin] PATH_INFO: `/v3/projects/4823ea9a83cd4fb5a3362bbcc71874ae/users/55a8bccd840c4eb0962700280cbd6e23/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:53.342185 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87677038-7c47-412e-9294-bb8a535ce9b8 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:53.345996 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2535e-0f05-443c-accd-33a1b8bee0ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:53.346796 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2535e-0f05-443c-accd-33a1b8bee0ad service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:53.347107 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2535e-0f05-443c-accd-33a1b8bee0ad service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:53.347328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf34321d-f1e1-4d61-ba56-921ec56ded3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4823ea9a83cd4fb5a3362bbcc71874ae, user_id=55a8bccd840c4eb0962700280cbd6e23, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:53.347468 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2535e-0f05-443c-accd-33a1b8bee0ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:53.350005 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf34321d-f1e1-4d61-ba56-921ec56ded3f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:53.363726 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2535e-0f05-443c-accd-33a1b8bee0ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:53.365409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2535e-0f05-443c-accd-33a1b8bee0ad service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:53.378161 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1046/2064] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:53.378776 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1019/2065] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:53 2022] PUT /identity/v3/projects/4823ea9a83cd4fb5a3362bbcc71874ae/users/55a8bccd840c4eb0962700280cbd6e23/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:53.379198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2535e-0f05-443c-accd-33a1b8bee0ad service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:53.382810 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34321d-f1e1-4d61-ba56-921ec56ded3f admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:53.387446 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:53.392232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2535e-0f05-443c-accd-33a1b8bee0ad service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:53.468723 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34321d-f1e1-4d61-ba56-921ec56ded3f admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:53.473974 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d482d-0b90-4524-ac62-53cfe5997404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:53.474619 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d482d-0b90-4524-ac62-53cfe5997404 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:53.474906 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d482d-0b90-4524-ac62-53cfe5997404 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:53.475269 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d482d-0b90-4524-ac62-53cfe5997404 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:53.476175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d482d-0b90-4524-ac62-53cfe5997404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:53.479453 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d482d-0b90-4524-ac62-53cfe5997404 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:53.484373 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b845e3b-aea6-44c6-9b1f-c3ee998be3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:53.485581 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b845e3b-aea6-44c6-9b1f-c3ee998be3dc admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:53.485948 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b845e3b-aea6-44c6-9b1f-c3ee998be3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:53.486225 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b845e3b-aea6-44c6-9b1f-c3ee998be3dc admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:53.487137 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b845e3b-aea6-44c6-9b1f-c3ee998be3dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:53.490034 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b845e3b-aea6-44c6-9b1f-c3ee998be3dc admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:53.500336 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1020/2066] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:53.507144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d482d-0b90-4524-ac62-53cfe5997404 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:53.521371 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d482d-0b90-4524-ac62-53cfe5997404 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:53.528044 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b9802-3c55-4368-b1fa-7320e79f6da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:53.529254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b9802-3c55-4368-b1fa-7320e79f6da6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:53.529833 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b9802-3c55-4368-b1fa-7320e79f6da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:53.530265 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b9802-3c55-4368-b1fa-7320e79f6da6 admin admin] PATH_INFO: `/v3/projects/4823ea9a83cd4fb5a3362bbcc71874ae/users/55a8bccd840c4eb0962700280cbd6e23/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:53.532495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767b9802-3c55-4368-b1fa-7320e79f6da6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4823ea9a83cd4fb5a3362bbcc71874ae, user_id=55a8bccd840c4eb0962700280cbd6e23, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:53.533626 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1047/2067] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:52:53 2022] POST /identity/v3/projects => generated 361 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:53.535389 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767b9802-3c55-4368-b1fa-7320e79f6da6 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:53.537961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b845e3b-aea6-44c6-9b1f-c3ee998be3dc admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:53.547789 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1021/2068] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:53 2022] PUT /identity/v3/projects/4823ea9a83cd4fb5a3362bbcc71874ae/users/55a8bccd840c4eb0962700280cbd6e23/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:52:53.558123 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8842e-d590-42f0-8342-d537e73b2631 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:53.558414 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8842e-d590-42f0-8342-d537e73b2631 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:53.558701 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8842e-d590-42f0-8342-d537e73b2631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:53.592383 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-3bf8842e-d590-42f0-8342-d537e73b2631 None None] MFA Rules not processed for user `55a8bccd840c4eb0962700280cbd6e23`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:53.593613 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3bf8842e-d590-42f0-8342-d537e73b2631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:53.601852 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:53.620287 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b845e3b-aea6-44c6-9b1f-c3ee998be3dc admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:53.693851 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a4448-92c8-449d-8fb8-0055463b1e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:53.694594 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a4448-92c8-449d-8fb8-0055463b1e4b admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:53.694871 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a4448-92c8-449d-8fb8-0055463b1e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:53.695175 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07a4448-92c8-449d-8fb8-0055463b1e4b admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:53.696613 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a4448-92c8-449d-8fb8-0055463b1e4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:53.697752 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07a4448-92c8-449d-8fb8-0055463b1e4b admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:53.703179 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1022/2069] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:53 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:53.713036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8842e-d590-42f0-8342-d537e73b2631 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:53.714553 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3bf8842e-d590-42f0-8342-d537e73b2631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:53.730702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1048/2070] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:52:53 2022] POST /identity/v3/users => generated 281 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:52:53.734224 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a4448-92c8-449d-8fb8-0055463b1e4b admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:53.736061 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b07a4448-92c8-449d-8fb8-0055463b1e4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:53.754786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07a4448-92c8-449d-8fb8-0055463b1e4b admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:53.761718 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687abb93-08f7-4a01-b966-f6fb9e8a5152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:53.762389 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687abb93-08f7-4a01-b966-f6fb9e8a5152 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:53.762669 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687abb93-08f7-4a01-b966-f6fb9e8a5152 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:53.763135 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687abb93-08f7-4a01-b966-f6fb9e8a5152 admin admin] PATH_INFO: `/v3/projects/aeeef876830c4789be1e72353771f258` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:53.764959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687abb93-08f7-4a01-b966-f6fb9e8a5152 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aeeef876830c4789be1e72353771f258)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:53.770946 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687abb93-08f7-4a01-b966-f6fb9e8a5152 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:53.786052 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8842e-d590-42f0-8342-d537e73b2631 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:53.790233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f435e4e-022e-4a22-9acc-c76b35a8028f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:53.790853 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f435e4e-022e-4a22-9acc-c76b35a8028f service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:53.791163 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f435e4e-022e-4a22-9acc-c76b35a8028f service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:53.791432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f435e4e-022e-4a22-9acc-c76b35a8028f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:53.793891 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:53.802867 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f435e4e-022e-4a22-9acc-c76b35a8028f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:53.804441 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f435e4e-022e-4a22-9acc-c76b35a8028f service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:53.815229 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1023/2071] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:53 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:53.815952 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f435e4e-022e-4a22-9acc-c76b35a8028f service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:53.827201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f435e4e-022e-4a22-9acc-c76b35a8028f service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:53.827647 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-687abb93-08f7-4a01-b966-f6fb9e8a5152 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeeef876830c4789be1e72353771f258', 'request_id': 'req-687abb93-08f7-4a01-b966-f6fb9e8a5152'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:53.842149 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-687abb93-08f7-4a01-b966-f6fb9e8a5152 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeeef876830c4789be1e72353771f258', 'request_id': 'req-687abb93-08f7-4a01-b966-f6fb9e8a5152'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:53.852094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-687abb93-08f7-4a01-b966-f6fb9e8a5152 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeeef876830c4789be1e72353771f258', 'request_id': 'req-687abb93-08f7-4a01-b966-f6fb9e8a5152'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:53.870116 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-687abb93-08f7-4a01-b966-f6fb9e8a5152 admin admin] The token cache is being invalidate because project aeeef876830c4789be1e72353771f258 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:52:53.870429 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-687abb93-08f7-4a01-b966-f6fb9e8a5152 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:53.871253 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-687abb93-08f7-4a01-b966-f6fb9e8a5152 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:53.875081 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1049/2072] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:52:53 2022] DELETE /identity/v3/projects/aeeef876830c4789be1e72353771f258 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:53.877287 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687abb93-08f7-4a01-b966-f6fb9e8a5152 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:53.878944 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-687abb93-08f7-4a01-b966-f6fb9e8a5152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:53.912506 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0632d-8f3a-4f7e-9a04-41d5cde817be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:53.913301 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0632d-8f3a-4f7e-9a04-41d5cde817be admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:53.913719 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0632d-8f3a-4f7e-9a04-41d5cde817be admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:53.914179 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0632d-8f3a-4f7e-9a04-41d5cde817be admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:53.914936 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b0632d-8f3a-4f7e-9a04-41d5cde817be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:53.916642 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b0632d-8f3a-4f7e-9a04-41d5cde817be admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:53.934918 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1024/2073] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:53.937135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0632d-8f3a-4f7e-9a04-41d5cde817be admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:53.939246 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c9b0632d-8f3a-4f7e-9a04-41d5cde817be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:53.954362 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0632d-8f3a-4f7e-9a04-41d5cde817be admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:53.960818 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da804c0-8a56-451d-95e7-500d5dcce517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:53.961466 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da804c0-8a56-451d-95e7-500d5dcce517 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:53.961799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da804c0-8a56-451d-95e7-500d5dcce517 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:53.962099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da804c0-8a56-451d-95e7-500d5dcce517 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:53.963157 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da804c0-8a56-451d-95e7-500d5dcce517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:53.964875 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da804c0-8a56-451d-95e7-500d5dcce517 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:53.966606 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687abb93-08f7-4a01-b966-f6fb9e8a5152 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:53.972329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec1d76-7318-4a48-b982-e70681f7e37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:53.973001 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec1d76-7318-4a48-b982-e70681f7e37f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:53.973279 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec1d76-7318-4a48-b982-e70681f7e37f admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:53.973727 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec1d76-7318-4a48-b982-e70681f7e37f admin admin] PATH_INFO: `/v3/users/e5058323532245d5afb4b8f3c8f176bd` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:53.974693 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec1d76-7318-4a48-b982-e70681f7e37f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5058323532245d5afb4b8f3c8f176bd)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:53.976730 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1025/2074] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:53.979019 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec1d76-7318-4a48-b982-e70681f7e37f admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:53.983828 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da804c0-8a56-451d-95e7-500d5dcce517 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:53.996676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da804c0-8a56-451d-95e7-500d5dcce517 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:54.003163 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e91023-859f-4638-8edc-cb1b99f3fd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:54.003852 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e91023-859f-4638-8edc-cb1b99f3fd10 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:54.004629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e91023-859f-4638-8edc-cb1b99f3fd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:54.004629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e91023-859f-4638-8edc-cb1b99f3fd10 admin admin] PATH_INFO: `/v3/projects/b3175e7b8d52440b8a065817b091e9f7/users/a958c2a0bdde4b76a5366d556548522f/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:54.028442 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e91023-859f-4638-8edc-cb1b99f3fd10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3175e7b8d52440b8a065817b091e9f7, user_id=a958c2a0bdde4b76a5366d556548522f, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:54.041183 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e91023-859f-4638-8edc-cb1b99f3fd10 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:54.047169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-3aec1d76-7318-4a48-b982-e70681f7e37f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5058323532245d5afb4b8f3c8f176bd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:54.071513 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1026/2075] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:53 2022] PUT /identity/v3/projects/b3175e7b8d52440b8a065817b091e9f7/users/a958c2a0bdde4b76a5366d556548522f/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:52:54.071513 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e91023-859f-4638-8edc-cb1b99f3fd10 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:54.071513 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-48e91023-859f-4638-8edc-cb1b99f3fd10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:54.072005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-3aec1d76-7318-4a48-b982-e70681f7e37f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5058323532245d5afb4b8f3c8f176bd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:54.075644 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-3aec1d76-7318-4a48-b982-e70681f7e37f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5058323532245d5afb4b8f3c8f176bd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:54.077746 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:54.088270 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-3aec1d76-7318-4a48-b982-e70681f7e37f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5058323532245d5afb4b8f3c8f176bd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:54.089025 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-3aec1d76-7318-4a48-b982-e70681f7e37f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5058323532245d5afb4b8f3c8f176bd'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:54.101827 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1050/2076] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:52:53 2022] DELETE /identity/v3/users/e5058323532245d5afb4b8f3c8f176bd => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:54.101827 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d4867-cae4-4f52-bb40-f6cfad76b821 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:54.101827 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d4867-cae4-4f52-bb40-f6cfad76b821 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:54.101827 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d4867-cae4-4f52-bb40-f6cfad76b821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:54.154639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-591d4867-cae4-4f52-bb40-f6cfad76b821 None None] MFA Rules not processed for user `a958c2a0bdde4b76a5366d556548522f`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:54.159222 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-591d4867-cae4-4f52-bb40-f6cfad76b821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:54.194556 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e91023-859f-4638-8edc-cb1b99f3fd10 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:54.196730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:54.299687 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1051/2077] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:54.336517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82ca4c-ffb9-4b4b-b922-19c3939dd113 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:54.338149 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82ca4c-ffb9-4b4b-b922-19c3939dd113 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:54.338755 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82ca4c-ffb9-4b4b-b922-19c3939dd113 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:54.339365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82ca4c-ffb9-4b4b-b922-19c3939dd113 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:54.345130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3d82ca4c-ffb9-4b4b-b922-19c3939dd113 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:54.359650 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d4867-cae4-4f52-bb40-f6cfad76b821 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:54.362206 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-591d4867-cae4-4f52-bb40-f6cfad76b821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:54.391477 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d4867-cae4-4f52-bb40-f6cfad76b821 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:54.396360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751cb9c-b7cc-4714-8d83-553603b693c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:54.397332 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751cb9c-b7cc-4714-8d83-553603b693c4 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:54.397692 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751cb9c-b7cc-4714-8d83-553603b693c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:54.399324 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751cb9c-b7cc-4714-8d83-553603b693c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:54.424521 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d751cb9c-b7cc-4714-8d83-553603b693c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:54.434924 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d751cb9c-b7cc-4714-8d83-553603b693c4 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:54.455593 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1052/2078] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:54.471838 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751cb9c-b7cc-4714-8d83-553603b693c4 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:54.474766 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d751cb9c-b7cc-4714-8d83-553603b693c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:54.498343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82ca4c-ffb9-4b4b-b922-19c3939dd113 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:54.500368 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82ca4c-ffb9-4b4b-b922-19c3939dd113 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:54.539116 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1027/2079] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:54 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:52:54.540897 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82ca4c-ffb9-4b4b-b922-19c3939dd113 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:54.542976 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3d82ca4c-ffb9-4b4b-b922-19c3939dd113 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:54.619540 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751cb9c-b7cc-4714-8d83-553603b693c4 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:54.626165 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea6fbe-edaf-4d78-8d69-973ae977f37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:54.627233 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea6fbe-edaf-4d78-8d69-973ae977f37f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:54.627510 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea6fbe-edaf-4d78-8d69-973ae977f37f admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:54.627937 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea6fbe-edaf-4d78-8d69-973ae977f37f admin admin] PATH_INFO: `/v3/users/a958c2a0bdde4b76a5366d556548522f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:54.629386 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea6fbe-edaf-4d78-8d69-973ae977f37f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a958c2a0bdde4b76a5366d556548522f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:54.631438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea6fbe-edaf-4d78-8d69-973ae977f37f admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:54.680926 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82ca4c-ffb9-4b4b-b922-19c3939dd113 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:54.689630 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a8c015-1a94-4fa0-ac24-95e523eb6e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:54.690631 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8c015-1a94-4fa0-ac24-95e523eb6e61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:54.691219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8c015-1a94-4fa0-ac24-95e523eb6e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:54.691514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8c015-1a94-4fa0-ac24-95e523eb6e61 admin admin] PATH_INFO: `/v3/projects/d81c1b6bde7b46e8ab03ea764e160655` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:54.693030 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a8c015-1a94-4fa0-ac24-95e523eb6e61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d81c1b6bde7b46e8ab03ea764e160655)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:54.695041 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a8c015-1a94-4fa0-ac24-95e523eb6e61 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:54.719749 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:54.721249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-78ea6fbe-edaf-4d78-8d69-973ae977f37f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a958c2a0bdde4b76a5366d556548522f'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:54.738574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-78ea6fbe-edaf-4d78-8d69-973ae977f37f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a958c2a0bdde4b76a5366d556548522f'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:54.750104 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-78ea6fbe-edaf-4d78-8d69-973ae977f37f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a958c2a0bdde4b76a5366d556548522f'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:54.759842 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e6a8c015-1a94-4fa0-ac24-95e523eb6e61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd81c1b6bde7b46e8ab03ea764e160655', 'request_id': 'req-e6a8c015-1a94-4fa0-ac24-95e523eb6e61'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:54.760486 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-78ea6fbe-edaf-4d78-8d69-973ae977f37f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a958c2a0bdde4b76a5366d556548522f'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:54.761369 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-78ea6fbe-edaf-4d78-8d69-973ae977f37f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a958c2a0bdde4b76a5366d556548522f'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:54.768865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e6a8c015-1a94-4fa0-ac24-95e523eb6e61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd81c1b6bde7b46e8ab03ea764e160655', 'request_id': 'req-e6a8c015-1a94-4fa0-ac24-95e523eb6e61'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:54.772373 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1053/2080] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:52:54 2022] DELETE /identity/v3/users/a958c2a0bdde4b76a5366d556548522f => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:54.774276 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679580c-0f8b-4d01-a0a5-21f9bee04333 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:54.774620 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679580c-0f8b-4d01-a0a5-21f9bee04333 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:54.774978 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679580c-0f8b-4d01-a0a5-21f9bee04333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:54.776095 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e6a8c015-1a94-4fa0-ac24-95e523eb6e61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd81c1b6bde7b46e8ab03ea764e160655', 'request_id': 'req-e6a8c015-1a94-4fa0-ac24-95e523eb6e61'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:54.788824 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e6a8c015-1a94-4fa0-ac24-95e523eb6e61 admin admin] The token cache is being invalidate because project d81c1b6bde7b46e8ab03ea764e160655 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:52:54.789113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e6a8c015-1a94-4fa0-ac24-95e523eb6e61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:54.789991 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e6a8c015-1a94-4fa0-ac24-95e523eb6e61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:54.791788 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1028/2081] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:52:54 2022] DELETE /identity/v3/projects/d81c1b6bde7b46e8ab03ea764e160655 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:54.798026 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f19d0-4e8e-4078-8010-1f9f2b6a5757 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:54.798439 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f19d0-4e8e-4078-8010-1f9f2b6a5757 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:54.798650 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f19d0-4e8e-4078-8010-1f9f2b6a5757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:54.800120 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-8679580c-0f8b-4d01-a0a5-21f9bee04333 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:54.801370 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8679580c-0f8b-4d01-a0a5-21f9bee04333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:54.820827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-209f19d0-4e8e-4078-8010-1f9f2b6a5757 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:54.822094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-209f19d0-4e8e-4078-8010-1f9f2b6a5757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:54.829754 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:54.889468 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1054/2082] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:54.900337 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679580c-0f8b-4d01-a0a5-21f9bee04333 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:54.901837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8679580c-0f8b-4d01-a0a5-21f9bee04333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:54.920980 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1029/2083] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:54.929366 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820c873-6e4b-44ae-8a9b-5a399ff0e34a None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:54.930053 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820c873-6e4b-44ae-8a9b-5a399ff0e34a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:54.930335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3820c873-6e4b-44ae-8a9b-5a399ff0e34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:54.968781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-3820c873-6e4b-44ae-8a9b-5a399ff0e34a None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:54.970261 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3820c873-6e4b-44ae-8a9b-5a399ff0e34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:54.979385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1030/2084] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:52:54.984214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3820c873-6e4b-44ae-8a9b-5a399ff0e34a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:54.986701 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679580c-0f8b-4d01-a0a5-21f9bee04333 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:54.991785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ca03c-f1e6-46e3-b7e3-651eaac83362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:54.992564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ca03c-f1e6-46e3-b7e3-651eaac83362 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:54.993058 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ca03c-f1e6-46e3-b7e3-651eaac83362 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:54.993397 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ca03c-f1e6-46e3-b7e3-651eaac83362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:54.994754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3820c873-6e4b-44ae-8a9b-5a399ff0e34a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:55.000257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0ee4b-8ff5-4ed6-b035-1e772283dda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:55.001081 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0ee4b-8ff5-4ed6-b035-1e772283dda8 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:55.001353 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0ee4b-8ff5-4ed6-b035-1e772283dda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:55.001613 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0ee4b-8ff5-4ed6-b035-1e772283dda8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:55.002333 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0ee4b-8ff5-4ed6-b035-1e772283dda8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:52:55.002644 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0ee4b-8ff5-4ed6-b035-1e772283dda8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:55.004321 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0ee4b-8ff5-4ed6-b035-1e772283dda8 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:55.004668 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ca03c-f1e6-46e3-b7e3-651eaac83362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:55.006148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ca03c-f1e6-46e3-b7e3-651eaac83362 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:55.021693 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1055/2085] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:55.027665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1031/2086] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 08:52:55.032986 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ca03c-f1e6-46e3-b7e3-651eaac83362 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:55.046060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ca03c-f1e6-46e3-b7e3-651eaac83362 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:55.051965 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd2572-296f-4b9d-b1e1-d1292c7abbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:55.052649 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd2572-296f-4b9d-b1e1-d1292c7abbcd admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:55.052946 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd2572-296f-4b9d-b1e1-d1292c7abbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:55.053222 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd2572-296f-4b9d-b1e1-d1292c7abbcd admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:55.053924 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd2572-296f-4b9d-b1e1-d1292c7abbcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:55.055948 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd2572-296f-4b9d-b1e1-d1292c7abbcd admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:55.091037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1056/2087] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:52:55 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:55.092759 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0ee4b-8ff5-4ed6-b035-1e772283dda8 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:55.104040 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0ee4b-8ff5-4ed6-b035-1e772283dda8 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:55.178052 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a75261-1f34-429d-b8c2-be0c63daf022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:55.178796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a75261-1f34-429d-b8c2-be0c63daf022 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:55.179220 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a75261-1f34-429d-b8c2-be0c63daf022 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:55.179438 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a75261-1f34-429d-b8c2-be0c63daf022 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:55.180086 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a75261-1f34-429d-b8c2-be0c63daf022 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:55.182053 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a75261-1f34-429d-b8c2-be0c63daf022 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:55.226418 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1032/2088] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:52:55 2022] POST /identity/v3/users => generated 287 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:55.228156 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd2572-296f-4b9d-b1e1-d1292c7abbcd admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:55.240803 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd2572-296f-4b9d-b1e1-d1292c7abbcd admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:55.247317 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a491c-00d2-4a0e-a371-653a672ad96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:55.247988 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a491c-00d2-4a0e-a371-653a672ad96c admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:55.248295 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a491c-00d2-4a0e-a371-653a672ad96c admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:55.248535 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a491c-00d2-4a0e-a371-653a672ad96c admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:55.249237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545a491c-00d2-4a0e-a371-653a672ad96c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:55.250829 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545a491c-00d2-4a0e-a371-653a672ad96c admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:55.262818 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1057/2089] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:55.267531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a75261-1f34-429d-b8c2-be0c63daf022 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:55.282218 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a75261-1f34-429d-b8c2-be0c63daf022 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:55.287750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4c910-20c0-4bba-a933-f334b4e26979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:55.288496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4c910-20c0-4bba-a933-f334b4e26979 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:55.288781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4c910-20c0-4bba-a933-f334b4e26979 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:55.289059 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4c910-20c0-4bba-a933-f334b4e26979 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:55.289765 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c4c910-20c0-4bba-a933-f334b4e26979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:55.291472 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c4c910-20c0-4bba-a933-f334b4e26979 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:55.305204 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1033/2090] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:52:55.309855 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a491c-00d2-4a0e-a371-653a672ad96c admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:55.321509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a491c-00d2-4a0e-a371-653a672ad96c admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:55.330249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07b0a1-7766-4b18-a67f-d150820c1d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:55.331284 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07b0a1-7766-4b18-a67f-d150820c1d33 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:55.331565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07b0a1-7766-4b18-a67f-d150820c1d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:55.331842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07b0a1-7766-4b18-a67f-d150820c1d33 admin admin] PATH_INFO: `/v3/projects/196b8882c5344c8380fecba536f39214/users/230efbe75687494fa44777325c8acb4e/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:55.367720 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c07b0a1-7766-4b18-a67f-d150820c1d33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=196b8882c5344c8380fecba536f39214, user_id=230efbe75687494fa44777325c8acb4e, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:55.373595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4c910-20c0-4bba-a933-f334b4e26979 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:55.375780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c07b0a1-7766-4b18-a67f-d150820c1d33 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:55.383764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f1c4c910-20c0-4bba-a933-f334b4e26979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:55.396152 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:55.402414 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4c910-20c0-4bba-a933-f334b4e26979 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:55.414283 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbea62d-12f9-4b5c-b6b6-f8e3a6d4b481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:55.415142 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbea62d-12f9-4b5c-b6b6-f8e3a6d4b481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:55.415430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbea62d-12f9-4b5c-b6b6-f8e3a6d4b481 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:55.416136 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbea62d-12f9-4b5c-b6b6-f8e3a6d4b481 admin admin] PATH_INFO: `/v3/projects/b3175e7b8d52440b8a065817b091e9f7` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:55.416918 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1058/2091] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:55 2022] PUT /identity/v3/projects/196b8882c5344c8380fecba536f39214/users/230efbe75687494fa44777325c8acb4e/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:55.417959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbea62d-12f9-4b5c-b6b6-f8e3a6d4b481 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3175e7b8d52440b8a065817b091e9f7)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:55.421406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbea62d-12f9-4b5c-b6b6-f8e3a6d4b481 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:55.422796 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d1efe-43f5-4a78-92e9-b9913f8ec265 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:55.422995 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d1efe-43f5-4a78-92e9-b9913f8ec265 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:55.423486 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d1efe-43f5-4a78-92e9-b9913f8ec265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:55.453893 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-919d1efe-43f5-4a78-92e9-b9913f8ec265 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:55.456442 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-919d1efe-43f5-4a78-92e9-b9913f8ec265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:55.504457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-6bbea62d-12f9-4b5c-b6b6-f8e3a6d4b481 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3175e7b8d52440b8a065817b091e9f7', 'request_id': 'req-6bbea62d-12f9-4b5c-b6b6-f8e3a6d4b481'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:55.519718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-6bbea62d-12f9-4b5c-b6b6-f8e3a6d4b481 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3175e7b8d52440b8a065817b091e9f7', 'request_id': 'req-6bbea62d-12f9-4b5c-b6b6-f8e3a6d4b481'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:55.527398 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-6bbea62d-12f9-4b5c-b6b6-f8e3a6d4b481 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3175e7b8d52440b8a065817b091e9f7', 'request_id': 'req-6bbea62d-12f9-4b5c-b6b6-f8e3a6d4b481'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:55.531553 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1059/2092] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:55.542245 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d1efe-43f5-4a78-92e9-b9913f8ec265 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:55.546638 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-6bbea62d-12f9-4b5c-b6b6-f8e3a6d4b481 admin admin] The token cache is being invalidate because project b3175e7b8d52440b8a065817b091e9f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:52:55.547312 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-6bbea62d-12f9-4b5c-b6b6-f8e3a6d4b481 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:55.548993 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-6bbea62d-12f9-4b5c-b6b6-f8e3a6d4b481 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:52:55.550691 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1034/2093] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:52:55 2022] DELETE /identity/v3/projects/b3175e7b8d52440b8a065817b091e9f7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:55.556117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bbb30-48c3-4f65-8c8d-4b9b5a44d035 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:55.556117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bbb30-48c3-4f65-8c8d-4b9b5a44d035 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:55.556117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bbb30-48c3-4f65-8c8d-4b9b5a44d035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:55.564484 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:55.594157 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-4a6bbb30-48c3-4f65-8c8d-4b9b5a44d035 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:55.595962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4a6bbb30-48c3-4f65-8c8d-4b9b5a44d035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:55.626415 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d1efe-43f5-4a78-92e9-b9913f8ec265 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:55.697383 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1035/2094] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:52:55.699266 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002a509-66eb-4c20-966a-308f7fe24ba7 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:55.699552 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002a509-66eb-4c20-966a-308f7fe24ba7 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:55.699881 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002a509-66eb-4c20-966a-308f7fe24ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:55.711931 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ee030-d47b-41ec-983e-67c55c81f959 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:55.712648 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ee030-d47b-41ec-983e-67c55c81f959 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:55.712992 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ee030-d47b-41ec-983e-67c55c81f959 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:55.713338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ee030-d47b-41ec-983e-67c55c81f959 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:55.716235 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8f2ee030-d47b-41ec-983e-67c55c81f959 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:55.724111 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c002a509-66eb-4c20-966a-308f7fe24ba7 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:55.725660 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c002a509-66eb-4c20-966a-308f7fe24ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:55.730139 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ee030-d47b-41ec-983e-67c55c81f959 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:55.731818 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ee030-d47b-41ec-983e-67c55c81f959 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:55.734918 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1036/2095] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:55.739433 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002a509-66eb-4c20-966a-308f7fe24ba7 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:55.744205 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1060/2096] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:52:55.749450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002a509-66eb-4c20-966a-308f7fe24ba7 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:55.754727 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c1f371-a8d8-4602-b9a9-ed34e81a50b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:55.755415 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1f371-a8d8-4602-b9a9-ed34e81a50b7 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:55.755729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1f371-a8d8-4602-b9a9-ed34e81a50b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:55.756062 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1f371-a8d8-4602-b9a9-ed34e81a50b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:55.756853 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c1f371-a8d8-4602-b9a9-ed34e81a50b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:52:55.757160 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c1f371-a8d8-4602-b9a9-ed34e81a50b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:55.759047 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c1f371-a8d8-4602-b9a9-ed34e81a50b7 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:55.774363 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1037/2097] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:52:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:52:55.779355 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ee030-d47b-41ec-983e-67c55c81f959 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:55.790426 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ee030-d47b-41ec-983e-67c55c81f959 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:55.796257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f02d2-b707-4dfa-bf41-6d025f93c864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:55.796984 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f02d2-b707-4dfa-bf41-6d025f93c864 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:55.797320 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f02d2-b707-4dfa-bf41-6d025f93c864 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:55.797459 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f02d2-b707-4dfa-bf41-6d025f93c864 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:55.798268 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f02d2-b707-4dfa-bf41-6d025f93c864 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:55.800222 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f02d2-b707-4dfa-bf41-6d025f93c864 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:55.832736 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1061/2098] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:52:55 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:55.837224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c1f371-a8d8-4602-b9a9-ed34e81a50b7 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:55.849019 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c1f371-a8d8-4602-b9a9-ed34e81a50b7 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:55.933307 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ab873-371e-40fd-80c3-bc339e3880c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:55.934810 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ab873-371e-40fd-80c3-bc339e3880c0 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:55.935226 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ab873-371e-40fd-80c3-bc339e3880c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:55.935618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ab873-371e-40fd-80c3-bc339e3880c0 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:55.936924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ab873-371e-40fd-80c3-bc339e3880c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:55.940521 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ab873-371e-40fd-80c3-bc339e3880c0 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:55.967141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:52:55.997967 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1038/2099] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:52:55 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:52:55.999211 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f02d2-b707-4dfa-bf41-6d025f93c864 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:56.010919 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f02d2-b707-4dfa-bf41-6d025f93c864 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:56.016321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ffaa2-c433-40d5-89d1-9ad243b342d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:56.017212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ffaa2-c433-40d5-89d1-9ad243b342d1 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:56.017531 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ffaa2-c433-40d5-89d1-9ad243b342d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:56.017959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ffaa2-c433-40d5-89d1-9ad243b342d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:56.018846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ffaa2-c433-40d5-89d1-9ad243b342d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:56.020483 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ffaa2-c433-40d5-89d1-9ad243b342d1 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:56.038999 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1062/2100] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 08:52:56.045404 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ab873-371e-40fd-80c3-bc339e3880c0 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:56.062122 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ab873-371e-40fd-80c3-bc339e3880c0 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:56.067296 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69459244-6532-40c1-a78a-1bc6af3d7303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:56.067928 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69459244-6532-40c1-a78a-1bc6af3d7303 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:56.068398 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69459244-6532-40c1-a78a-1bc6af3d7303 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:56.069338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69459244-6532-40c1-a78a-1bc6af3d7303 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:56.069910 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69459244-6532-40c1-a78a-1bc6af3d7303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:56.071197 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69459244-6532-40c1-a78a-1bc6af3d7303 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:56.085707 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1039/2101] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 08:52:56.089646 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ffaa2-c433-40d5-89d1-9ad243b342d1 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:56.110596 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ffaa2-c433-40d5-89d1-9ad243b342d1 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:56.122595 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b666a-ed13-4c43-a733-c94209e9f5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:56.124182 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b666a-ed13-4c43-a733-c94209e9f5e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:56.124809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b666a-ed13-4c43-a733-c94209e9f5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:56.125126 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b666a-ed13-4c43-a733-c94209e9f5e7 admin admin] PATH_INFO: `/v3/projects/31ba3d19c3a54127a9fb527325d1806b/users/c55f66fd7b4e40e48108f5d378620ff4/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:56.157675 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b666a-ed13-4c43-a733-c94209e9f5e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31ba3d19c3a54127a9fb527325d1806b, user_id=c55f66fd7b4e40e48108f5d378620ff4, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:56.160790 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b666a-ed13-4c43-a733-c94209e9f5e7 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:56.178391 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1063/2102] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:52:56 2022] PUT /identity/v3/projects/31ba3d19c3a54127a9fb527325d1806b/users/c55f66fd7b4e40e48108f5d378620ff4/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:52:56.190716 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa69ac7-396b-4532-bdc2-001d3d886aaf None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:56.191257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa69ac7-396b-4532-bdc2-001d3d886aaf None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:56.191568 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa69ac7-396b-4532-bdc2-001d3d886aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:56.255294 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-9fa69ac7-396b-4532-bdc2-001d3d886aaf None None] MFA Rules not processed for user `c55f66fd7b4e40e48108f5d378620ff4`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:56.256580 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9fa69ac7-396b-4532-bdc2-001d3d886aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:56.412124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1040/2103] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:52:56.443094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b666a-ed13-4c43-a733-c94209e9f5e7 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:56.445160 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-243b666a-ed13-4c43-a733-c94209e9f5e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:56.522758 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b666a-ed13-4c43-a733-c94209e9f5e7 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:56.526451 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed4962-df83-4eab-9a09-85ef8482bb32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:56.527117 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed4962-df83-4eab-9a09-85ef8482bb32 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:56.527391 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed4962-df83-4eab-9a09-85ef8482bb32 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:56.527657 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed4962-df83-4eab-9a09-85ef8482bb32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:56.539793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed4962-df83-4eab-9a09-85ef8482bb32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:56.541336 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed4962-df83-4eab-9a09-85ef8482bb32 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:56.551343 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1064/2104] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:52:56.796796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c467f-348a-4fe8-ae2f-a044de15ddc0 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:56.797704 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c467f-348a-4fe8-ae2f-a044de15ddc0 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:56.797704 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c467f-348a-4fe8-ae2f-a044de15ddc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:56.843365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-e15c467f-348a-4fe8-ae2f-a044de15ddc0 None None] MFA Rules not processed for user `230efbe75687494fa44777325c8acb4e`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:52:56.845131 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e15c467f-348a-4fe8-ae2f-a044de15ddc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:52:56.970911 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1041/2105] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:52:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:52:56.987299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed4962-df83-4eab-9a09-85ef8482bb32 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:52:56.998607 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed4962-df83-4eab-9a09-85ef8482bb32 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:52:57.004871 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b14df-a218-4bc4-a723-2143fafc8608 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:52:57.006343 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b14df-a218-4bc4-a723-2143fafc8608 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:52:57.006859 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b14df-a218-4bc4-a723-2143fafc8608 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:52:57.007649 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b14df-a218-4bc4-a723-2143fafc8608 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:52:57.021308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64b14df-a218-4bc4-a723-2143fafc8608 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:52:57.023001 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64b14df-a218-4bc4-a723-2143fafc8608 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:52:57.035193 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1065/2106] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:52:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:53:01.824354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256d86a-1b62-440a-bf17-1c8eda225dcd None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:01.824354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256d86a-1b62-440a-bf17-1c8eda225dcd None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:01.825012 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256d86a-1b62-440a-bf17-1c8eda225dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:01.852336 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-1256d86a-1b62-440a-bf17-1c8eda225dcd None None] MFA Rules not processed for user `230efbe75687494fa44777325c8acb4e`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:01.853565 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1256d86a-1b62-440a-bf17-1c8eda225dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:01.861557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1042/2107] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:53:04.748996 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b14df-a218-4bc4-a723-2143fafc8608 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:04.751067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f64b14df-a218-4bc4-a723-2143fafc8608 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:04.761330 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:53:04.923148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b14df-a218-4bc4-a723-2143fafc8608 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:04.938144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c5cb3-1b39-4e38-a8fb-86019737c466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:04.940153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c5cb3-1b39-4e38-a8fb-86019737c466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:04.941609 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c5cb3-1b39-4e38-a8fb-86019737c466 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:04.942773 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c5cb3-1b39-4e38-a8fb-86019737c466 admin admin] PATH_INFO: `/v3/users/4f66ad0b219843ceb7c33aa9cbc58a34` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:04.945827 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c5cb3-1b39-4e38-a8fb-86019737c466 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f66ad0b219843ceb7c33aa9cbc58a34)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:04.950297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987c5cb3-1b39-4e38-a8fb-86019737c466 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:05.079839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-987c5cb3-1b39-4e38-a8fb-86019737c466 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f66ad0b219843ceb7c33aa9cbc58a34'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:05.128572 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-987c5cb3-1b39-4e38-a8fb-86019737c466 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f66ad0b219843ceb7c33aa9cbc58a34'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:05.151541 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-987c5cb3-1b39-4e38-a8fb-86019737c466 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f66ad0b219843ceb7c33aa9cbc58a34'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:05.183911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-987c5cb3-1b39-4e38-a8fb-86019737c466 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f66ad0b219843ceb7c33aa9cbc58a34'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:05.186475 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-987c5cb3-1b39-4e38-a8fb-86019737c466 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f66ad0b219843ceb7c33aa9cbc58a34'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:05.212556 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1066/2108] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:53:04 2022] DELETE /identity/v3/users/4f66ad0b219843ceb7c33aa9cbc58a34 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:53:05.216838 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d6661-b361-49da-9ebe-66e7e7e242f8 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:05.217856 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d6661-b361-49da-9ebe-66e7e7e242f8 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:05.217856 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d6661-b361-49da-9ebe-66e7e7e242f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:05.258666 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-922d6661-b361-49da-9ebe-66e7e7e242f8 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:05.260874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-922d6661-b361-49da-9ebe-66e7e7e242f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:05.356551 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:53:05.384761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1043/2109] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:53:05.400107 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c5cb3-1b39-4e38-a8fb-86019737c466 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:05.401769 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-987c5cb3-1b39-4e38-a8fb-86019737c466 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:05.504825 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c5cb3-1b39-4e38-a8fb-86019737c466 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:05.509592 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9562a-0c03-4e18-98d9-69a1411ee42a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:05.510230 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9562a-0c03-4e18-98d9-69a1411ee42a service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:05.510509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9562a-0c03-4e18-98d9-69a1411ee42a service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:05.510776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff9562a-0c03-4e18-98d9-69a1411ee42a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:05.523122 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff9562a-0c03-4e18-98d9-69a1411ee42a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:05.525000 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff9562a-0c03-4e18-98d9-69a1411ee42a service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:05.539332 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1067/2110] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:53:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:53:06.003419 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d6661-b361-49da-9ebe-66e7e7e242f8 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:06.005436 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-922d6661-b361-49da-9ebe-66e7e7e242f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:06.037059 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d6661-b361-49da-9ebe-66e7e7e242f8 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:06.052324 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f4c68-cdc1-4234-8b24-59925414d5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:06.053418 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f4c68-cdc1-4234-8b24-59925414d5f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:06.053971 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f4c68-cdc1-4234-8b24-59925414d5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:06.054387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f4c68-cdc1-4234-8b24-59925414d5f7 admin admin] PATH_INFO: `/v3/projects/d013d9db6425442293db93fb16b172cf` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:06.057523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f4c68-cdc1-4234-8b24-59925414d5f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d013d9db6425442293db93fb16b172cf)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:06.061358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f4c68-cdc1-4234-8b24-59925414d5f7 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:06.193707 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0d6f4c68-cdc1-4234-8b24-59925414d5f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd013d9db6425442293db93fb16b172cf', 'request_id': 'req-0d6f4c68-cdc1-4234-8b24-59925414d5f7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:06.207496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0d6f4c68-cdc1-4234-8b24-59925414d5f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd013d9db6425442293db93fb16b172cf', 'request_id': 'req-0d6f4c68-cdc1-4234-8b24-59925414d5f7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:06.242265 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0d6f4c68-cdc1-4234-8b24-59925414d5f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd013d9db6425442293db93fb16b172cf', 'request_id': 'req-0d6f4c68-cdc1-4234-8b24-59925414d5f7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:06.270611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0d6f4c68-cdc1-4234-8b24-59925414d5f7 admin admin] The token cache is being invalidate because project d013d9db6425442293db93fb16b172cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:53:06.271611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0d6f4c68-cdc1-4234-8b24-59925414d5f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:06.272673 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0d6f4c68-cdc1-4234-8b24-59925414d5f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:06.275048 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1044/2111] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:53:06 2022] DELETE /identity/v3/projects/d013d9db6425442293db93fb16b172cf => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:53:06.293752 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9562a-0c03-4e18-98d9-69a1411ee42a service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:06.297240 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff9562a-0c03-4e18-98d9-69a1411ee42a service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:06.377159 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:53:06.445378 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426ac73-636d-476b-a437-511db4be31de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:06.446141 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426ac73-636d-476b-a437-511db4be31de admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:06.446500 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426ac73-636d-476b-a437-511db4be31de admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:06.446837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426ac73-636d-476b-a437-511db4be31de admin admin] PATH_INFO: `/v3/users/55a8bccd840c4eb0962700280cbd6e23` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:06.455820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6426ac73-636d-476b-a437-511db4be31de admin admin] RBAC: Authorizing `identity:delete_user(user_id=55a8bccd840c4eb0962700280cbd6e23)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:06.457875 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6426ac73-636d-476b-a437-511db4be31de admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:06.540763 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6426ac73-636d-476b-a437-511db4be31de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55a8bccd840c4eb0962700280cbd6e23'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:06.555740 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6426ac73-636d-476b-a437-511db4be31de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55a8bccd840c4eb0962700280cbd6e23'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:06.565645 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6426ac73-636d-476b-a437-511db4be31de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55a8bccd840c4eb0962700280cbd6e23'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:06.596067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6426ac73-636d-476b-a437-511db4be31de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55a8bccd840c4eb0962700280cbd6e23'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:06.599128 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6426ac73-636d-476b-a437-511db4be31de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55a8bccd840c4eb0962700280cbd6e23'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:06.608549 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1068/2112] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:53:06 2022] DELETE /identity/v3/users/55a8bccd840c4eb0962700280cbd6e23 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:53:06.947053 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f4c68-cdc1-4234-8b24-59925414d5f7 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:06.948624 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0d6f4c68-cdc1-4234-8b24-59925414d5f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:06.964445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f4c68-cdc1-4234-8b24-59925414d5f7 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:07.096870 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed854d55-23d9-46c6-bb94-edd183168ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:07.097423 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed854d55-23d9-46c6-bb94-edd183168ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:07.097700 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed854d55-23d9-46c6-bb94-edd183168ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:07.098186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed854d55-23d9-46c6-bb94-edd183168ed7 admin admin] PATH_INFO: `/v3/projects/4823ea9a83cd4fb5a3362bbcc71874ae` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:07.099404 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed854d55-23d9-46c6-bb94-edd183168ed7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4823ea9a83cd4fb5a3362bbcc71874ae)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:07.101483 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed854d55-23d9-46c6-bb94-edd183168ed7 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:07.166075 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ed854d55-23d9-46c6-bb94-edd183168ed7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4823ea9a83cd4fb5a3362bbcc71874ae', 'request_id': 'req-ed854d55-23d9-46c6-bb94-edd183168ed7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:07.177302 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ed854d55-23d9-46c6-bb94-edd183168ed7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4823ea9a83cd4fb5a3362bbcc71874ae', 'request_id': 'req-ed854d55-23d9-46c6-bb94-edd183168ed7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:07.192295 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ed854d55-23d9-46c6-bb94-edd183168ed7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4823ea9a83cd4fb5a3362bbcc71874ae', 'request_id': 'req-ed854d55-23d9-46c6-bb94-edd183168ed7'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:07.209044 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ed854d55-23d9-46c6-bb94-edd183168ed7 admin admin] The token cache is being invalidate because project 4823ea9a83cd4fb5a3362bbcc71874ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:53:07.209507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ed854d55-23d9-46c6-bb94-edd183168ed7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:07.213024 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-ed854d55-23d9-46c6-bb94-edd183168ed7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:07.214809 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1045/2113] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:53:06 2022] DELETE /identity/v3/projects/4823ea9a83cd4fb5a3362bbcc71874ae => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:53:07.227064 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdb4b2-c7a7-4008-b924-85541b665384 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:07.227346 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdb4b2-c7a7-4008-b924-85541b665384 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:07.227616 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdb4b2-c7a7-4008-b924-85541b665384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:07.250439 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-40bdb4b2-c7a7-4008-b924-85541b665384 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:07.251738 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-40bdb4b2-c7a7-4008-b924-85541b665384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:07.335812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1069/2114] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:53:07.336730 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03904732-506e-434a-86ff-961e5321179a None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:07.337235 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03904732-506e-434a-86ff-961e5321179a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:07.337341 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03904732-506e-434a-86ff-961e5321179a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:07.365536 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-03904732-506e-434a-86ff-961e5321179a None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:07.367613 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-03904732-506e-434a-86ff-961e5321179a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:07.377831 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1046/2115] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:53:07.387445 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bdb4b2-c7a7-4008-b924-85541b665384 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:07.399843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bdb4b2-c7a7-4008-b924-85541b665384 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:07.405786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d30597a-03e6-46b8-9ca1-5b26392d2236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:07.406439 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30597a-03e6-46b8-9ca1-5b26392d2236 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:07.406715 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30597a-03e6-46b8-9ca1-5b26392d2236 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:07.407036 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30597a-03e6-46b8-9ca1-5b26392d2236 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:07.407871 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d30597a-03e6-46b8-9ca1-5b26392d2236 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:53:07.408111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d30597a-03e6-46b8-9ca1-5b26392d2236 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:07.409776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d30597a-03e6-46b8-9ca1-5b26392d2236 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:07.426316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1070/2116] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:53:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:53:07.430665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03904732-506e-434a-86ff-961e5321179a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:07.446416 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03904732-506e-434a-86ff-961e5321179a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:07.452002 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120db5e7-3dff-4153-a90b-f1c1025f932a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:07.452925 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120db5e7-3dff-4153-a90b-f1c1025f932a admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:07.453621 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120db5e7-3dff-4153-a90b-f1c1025f932a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:07.454508 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120db5e7-3dff-4153-a90b-f1c1025f932a admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:07.455870 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120db5e7-3dff-4153-a90b-f1c1025f932a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:07.458152 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120db5e7-3dff-4153-a90b-f1c1025f932a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:07.463406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:53:07.528361 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1047/2117] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:53:07 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:53:07.533967 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d30597a-03e6-46b8-9ca1-5b26392d2236 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:07.548442 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d30597a-03e6-46b8-9ca1-5b26392d2236 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:07.617280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:53:07.684435 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ec524-2b82-49b8-9966-f749678b4856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:07.684435 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ec524-2b82-49b8-9966-f749678b4856 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:07.684435 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ec524-2b82-49b8-9966-f749678b4856 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:07.684435 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ec524-2b82-49b8-9966-f749678b4856 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:07.684778 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ec524-2b82-49b8-9966-f749678b4856 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:07.685849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ec524-2b82-49b8-9966-f749678b4856 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:07.734953 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1071/2118] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:53:07 2022] POST /identity/v3/users => generated 288 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:53:07.743217 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120db5e7-3dff-4153-a90b-f1c1025f932a admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:07.781135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120db5e7-3dff-4153-a90b-f1c1025f932a admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:07.798622 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161b7e1-6a44-4841-b29a-b0fe326d4253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:07.798622 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161b7e1-6a44-4841-b29a-b0fe326d4253 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:07.798622 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161b7e1-6a44-4841-b29a-b0fe326d4253 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:07.798622 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161b7e1-6a44-4841-b29a-b0fe326d4253 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:07.799131 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4161b7e1-6a44-4841-b29a-b0fe326d4253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:07.800418 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4161b7e1-6a44-4841-b29a-b0fe326d4253 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:07.817866 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1048/2119] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:53:07.825239 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ec524-2b82-49b8-9966-f749678b4856 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:07.853564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ec524-2b82-49b8-9966-f749678b4856 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:07.872110 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afacecbb-e00c-4568-9eb5-277af939c36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:07.873278 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afacecbb-e00c-4568-9eb5-277af939c36c admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:07.873278 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afacecbb-e00c-4568-9eb5-277af939c36c admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:07.873761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afacecbb-e00c-4568-9eb5-277af939c36c admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:07.875007 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afacecbb-e00c-4568-9eb5-277af939c36c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:07.876912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afacecbb-e00c-4568-9eb5-277af939c36c admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:07.914684 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1072/2120] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:53:07.915190 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161b7e1-6a44-4841-b29a-b0fe326d4253 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:07.930809 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161b7e1-6a44-4841-b29a-b0fe326d4253 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:07.937307 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e49058-a39c-4613-b66f-4f8223d67189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:07.938068 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e49058-a39c-4613-b66f-4f8223d67189 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:07.938274 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e49058-a39c-4613-b66f-4f8223d67189 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:07.938526 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e49058-a39c-4613-b66f-4f8223d67189 admin admin] PATH_INFO: `/v3/projects/7d68f425160c400ab4ee24a4be3fd9e2/users/9af14fae00dc4456aeef1ee27e8a3076/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:07.967046 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e49058-a39c-4613-b66f-4f8223d67189 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d68f425160c400ab4ee24a4be3fd9e2, user_id=9af14fae00dc4456aeef1ee27e8a3076, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:07.967762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e49058-a39c-4613-b66f-4f8223d67189 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:07.997047 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1049/2121] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:53:07 2022] PUT /identity/v3/projects/7d68f425160c400ab4ee24a4be3fd9e2/users/9af14fae00dc4456aeef1ee27e8a3076/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:53:08.009722 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd561f-3ddc-4275-a3e5-30c96d0a1365 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:08.010281 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd561f-3ddc-4275-a3e5-30c96d0a1365 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:08.010438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd561f-3ddc-4275-a3e5-30c96d0a1365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:08.042541 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-26cd561f-3ddc-4275-a3e5-30c96d0a1365 None None] MFA Rules not processed for user `9af14fae00dc4456aeef1ee27e8a3076`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:08.043832 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-26cd561f-3ddc-4275-a3e5-30c96d0a1365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:08.141993 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1073/2122] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:53:08.155135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e49058-a39c-4613-b66f-4f8223d67189 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:08.157077 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-66e49058-a39c-4613-b66f-4f8223d67189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:08.261157 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e49058-a39c-4613-b66f-4f8223d67189 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:08.274231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b0767e-8d26-49b8-9c29-b1d97d65bb21 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d178c717a3e49319545e70cb58cbbed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:08.275993 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0767e-8d26-49b8-9c29-b1d97d65bb21 service glance] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:08.276708 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0767e-8d26-49b8-9c29-b1d97d65bb21 service glance] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:08.277341 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0767e-8d26-49b8-9c29-b1d97d65bb21 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:08.300095 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b0767e-8d26-49b8-9c29-b1d97d65bb21 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:08.304196 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b0767e-8d26-49b8-9c29-b1d97d65bb21 service glance] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:08.337559 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1050/2123] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:53:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:53:11.864534 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd561f-3ddc-4275-a3e5-30c96d0a1365 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:11.866916 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-26cd561f-3ddc-4275-a3e5-30c96d0a1365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:11.973061 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd561f-3ddc-4275-a3e5-30c96d0a1365 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:11.986210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f004f-1a91-41e3-bc8d-8821055c285d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:11.987329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f004f-1a91-41e3-bc8d-8821055c285d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:11.987797 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f004f-1a91-41e3-bc8d-8821055c285d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:11.988408 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f004f-1a91-41e3-bc8d-8821055c285d admin admin] PATH_INFO: `/v3/users/c55f66fd7b4e40e48108f5d378620ff4` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:11.990059 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f004f-1a91-41e3-bc8d-8821055c285d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c55f66fd7b4e40e48108f5d378620ff4)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:11.993425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760f004f-1a91-41e3-bc8d-8821055c285d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:12.073894 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:53:12.086354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-760f004f-1a91-41e3-bc8d-8821055c285d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c55f66fd7b4e40e48108f5d378620ff4'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:12.116227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-760f004f-1a91-41e3-bc8d-8821055c285d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c55f66fd7b4e40e48108f5d378620ff4'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:12.131898 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-760f004f-1a91-41e3-bc8d-8821055c285d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c55f66fd7b4e40e48108f5d378620ff4'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:12.148738 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-760f004f-1a91-41e3-bc8d-8821055c285d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c55f66fd7b4e40e48108f5d378620ff4'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:12.149631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-760f004f-1a91-41e3-bc8d-8821055c285d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c55f66fd7b4e40e48108f5d378620ff4'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:12.157662 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1074/2124] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:53:11 2022] DELETE /identity/v3/users/c55f66fd7b4e40e48108f5d378620ff4 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:53:12.167051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c105dac-aea4-465c-a56c-9974be9dfde9 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:12.174651 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c105dac-aea4-465c-a56c-9974be9dfde9 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:12.174651 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c105dac-aea4-465c-a56c-9974be9dfde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:12.200084 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-0c105dac-aea4-465c-a56c-9974be9dfde9 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:12.202592 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0c105dac-aea4-465c-a56c-9974be9dfde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:12.381945 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1051/2125] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:53:12.408267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f004f-1a91-41e3-bc8d-8821055c285d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:12.410567 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-760f004f-1a91-41e3-bc8d-8821055c285d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:12.651280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f004f-1a91-41e3-bc8d-8821055c285d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:12.659263 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2449c34-fd48-4342-8a9d-dd63d76698dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:12.660380 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2449c34-fd48-4342-8a9d-dd63d76698dd service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:12.660877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2449c34-fd48-4342-8a9d-dd63d76698dd service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:12.661353 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2449c34-fd48-4342-8a9d-dd63d76698dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:12.695318 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2449c34-fd48-4342-8a9d-dd63d76698dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:12.697219 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2449c34-fd48-4342-8a9d-dd63d76698dd service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:12.730130 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1075/2126] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:53:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:53:14.046804 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c105dac-aea4-465c-a56c-9974be9dfde9 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:14.047718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0c105dac-aea4-465c-a56c-9974be9dfde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:14.074989 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c105dac-aea4-465c-a56c-9974be9dfde9 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:14.087042 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219a2a55-483b-4a67-a7ab-c5d92ae9683a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:14.088122 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a2a55-483b-4a67-a7ab-c5d92ae9683a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:14.088593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a2a55-483b-4a67-a7ab-c5d92ae9683a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:14.089126 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a2a55-483b-4a67-a7ab-c5d92ae9683a admin admin] PATH_INFO: `/v3/projects/31ba3d19c3a54127a9fb527325d1806b` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:14.091469 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219a2a55-483b-4a67-a7ab-c5d92ae9683a admin admin] RBAC: Authorizing `identity:delete_project(project_id=31ba3d19c3a54127a9fb527325d1806b)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:14.095816 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219a2a55-483b-4a67-a7ab-c5d92ae9683a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:14.260129 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:53:14.293206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-219a2a55-483b-4a67-a7ab-c5d92ae9683a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31ba3d19c3a54127a9fb527325d1806b', 'request_id': 'req-219a2a55-483b-4a67-a7ab-c5d92ae9683a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:14.326569 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-219a2a55-483b-4a67-a7ab-c5d92ae9683a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31ba3d19c3a54127a9fb527325d1806b', 'request_id': 'req-219a2a55-483b-4a67-a7ab-c5d92ae9683a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:14.377018 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-219a2a55-483b-4a67-a7ab-c5d92ae9683a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31ba3d19c3a54127a9fb527325d1806b', 'request_id': 'req-219a2a55-483b-4a67-a7ab-c5d92ae9683a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:14.437756 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-219a2a55-483b-4a67-a7ab-c5d92ae9683a admin admin] The token cache is being invalidate because project 31ba3d19c3a54127a9fb527325d1806b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:53:14.438273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-219a2a55-483b-4a67-a7ab-c5d92ae9683a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:14.447249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-219a2a55-483b-4a67-a7ab-c5d92ae9683a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:14.452892 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1052/2127] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:53:14 2022] DELETE /identity/v3/projects/31ba3d19c3a54127a9fb527325d1806b => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:53:15.502344 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772aea02-2497-4ee7-af1e-505c20afec32 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:15.503346 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772aea02-2497-4ee7-af1e-505c20afec32 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:15.503346 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772aea02-2497-4ee7-af1e-505c20afec32 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:15.505160 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1076/2128] 10.222.0.71 () {54 vars in 921 bytes} [Fri Aug 26 08:53:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:53:15.516010 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd0658-4bbc-43cf-8744-b833bb7face6 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:15.516930 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd0658-4bbc-43cf-8744-b833bb7face6 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:15.516930 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd0658-4bbc-43cf-8744-b833bb7face6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:15.567668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-a3dd0658-4bbc-43cf-8744-b833bb7face6 None None] MFA Rules not processed for user `0cf5e2b4f5e440a7942adea3d51cec75`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:15.570184 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a3dd0658-4bbc-43cf-8744-b833bb7face6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:15.635548 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772aea02-2497-4ee7-af1e-505c20afec32 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:15.640083 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-772aea02-2497-4ee7-af1e-505c20afec32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:15.734677 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1053/2129] 10.222.0.71 () {58 vars in 1018 bytes} [Fri Aug 26 08:53:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:53:15.757789 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a8770-3aa1-4d6f-8049-7b878b3241c0 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:15.758783 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a8770-3aa1-4d6f-8049-7b878b3241c0 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:15.760262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a8770-3aa1-4d6f-8049-7b878b3241c0 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:15.764114 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1054/2130] 10.222.0.71 () {54 vars in 961 bytes} [Fri Aug 26 08:53:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:53:15.785569 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14155035-0183-417a-9c00-c32e0f2b20e2 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:15.787726 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14155035-0183-417a-9c00-c32e0f2b20e2 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:15.788269 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14155035-0183-417a-9c00-c32e0f2b20e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:15.827585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-14155035-0183-417a-9c00-c32e0f2b20e2 None None] MFA Rules not processed for user `0cf5e2b4f5e440a7942adea3d51cec75`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:15.829665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-14155035-0183-417a-9c00-c32e0f2b20e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:15.839338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772aea02-2497-4ee7-af1e-505c20afec32 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:15.848288 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1055/2131] 10.222.0.71 () {58 vars in 1058 bytes} [Fri Aug 26 08:53:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:53:15.857383 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69732295-ba30-4f8c-91f0-d9c7e07b8476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:15.858878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69732295-ba30-4f8c-91f0-d9c7e07b8476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:15.859527 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69732295-ba30-4f8c-91f0-d9c7e07b8476 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:15.860090 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69732295-ba30-4f8c-91f0-d9c7e07b8476 admin admin] PATH_INFO: `/v3/users/d065c8a972df4b8e92996ce27e96b486` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:15.862925 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69732295-ba30-4f8c-91f0-d9c7e07b8476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d065c8a972df4b8e92996ce27e96b486)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:15.867968 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69732295-ba30-4f8c-91f0-d9c7e07b8476 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:15.872489 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14155035-0183-417a-9c00-c32e0f2b20e2 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:15.897484 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14155035-0183-417a-9c00-c32e0f2b20e2 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:15.910165 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55f6c8-649e-42d3-90d1-d11b29c612ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf5e2b4f5e440a7942adea3d51cec75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:15.911405 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55f6c8-649e-42d3-90d1-d11b29c612ff service placement] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:15.911825 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55f6c8-649e-42d3-90d1-d11b29c612ff service placement] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:15.912286 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55f6c8-649e-42d3-90d1-d11b29c612ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:15.935213 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55f6c8-649e-42d3-90d1-d11b29c612ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:15.937666 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55f6c8-649e-42d3-90d1-d11b29c612ff service placement] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:15.970386 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1056/2132] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:53:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:53:15.989024 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-69732295-ba30-4f8c-91f0-d9c7e07b8476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd065c8a972df4b8e92996ce27e96b486'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:16.027397 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:53:16.032118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-69732295-ba30-4f8c-91f0-d9c7e07b8476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd065c8a972df4b8e92996ce27e96b486'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:16.057257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-69732295-ba30-4f8c-91f0-d9c7e07b8476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd065c8a972df4b8e92996ce27e96b486'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:16.081227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-69732295-ba30-4f8c-91f0-d9c7e07b8476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd065c8a972df4b8e92996ce27e96b486'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:16.091464 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-69732295-ba30-4f8c-91f0-d9c7e07b8476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd065c8a972df4b8e92996ce27e96b486'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:16.108157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1077/2133] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:53:15 2022] DELETE /identity/v3/users/d065c8a972df4b8e92996ce27e96b486 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:53:16.413521 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55f6c8-649e-42d3-90d1-d11b29c612ff service placement] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:16.442880 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55f6c8-649e-42d3-90d1-d11b29c612ff service placement] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:16.549406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d224aff2-9157-477e-a848-046d5b32485e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:16.550411 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224aff2-9157-477e-a848-046d5b32485e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:16.550411 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224aff2-9157-477e-a848-046d5b32485e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:16.550718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224aff2-9157-477e-a848-046d5b32485e admin admin] PATH_INFO: `/v3/projects/76bb70b9748b4ed69bfc098327cc1234` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:16.551939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224aff2-9157-477e-a848-046d5b32485e admin admin] RBAC: Authorizing `identity:delete_project(project_id=76bb70b9748b4ed69bfc098327cc1234)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:16.553966 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d224aff2-9157-477e-a848-046d5b32485e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:16.627490 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d224aff2-9157-477e-a848-046d5b32485e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76bb70b9748b4ed69bfc098327cc1234', 'request_id': 'req-d224aff2-9157-477e-a848-046d5b32485e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:16.633844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:53:16.642020 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d224aff2-9157-477e-a848-046d5b32485e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76bb70b9748b4ed69bfc098327cc1234', 'request_id': 'req-d224aff2-9157-477e-a848-046d5b32485e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:16.655456 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d224aff2-9157-477e-a848-046d5b32485e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76bb70b9748b4ed69bfc098327cc1234', 'request_id': 'req-d224aff2-9157-477e-a848-046d5b32485e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:16.675340 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d224aff2-9157-477e-a848-046d5b32485e admin admin] The token cache is being invalidate because project 76bb70b9748b4ed69bfc098327cc1234 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:53:16.675869 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d224aff2-9157-477e-a848-046d5b32485e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:16.677143 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d224aff2-9157-477e-a848-046d5b32485e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:16.680352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1057/2134] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:53:16 2022] DELETE /identity/v3/projects/76bb70b9748b4ed69bfc098327cc1234 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:53:16.686619 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2caa96-7ece-4f94-b87b-3bf0c5dcb78f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:16.687463 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2caa96-7ece-4f94-b87b-3bf0c5dcb78f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:16.687679 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2caa96-7ece-4f94-b87b-3bf0c5dcb78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:16.730964 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-6d2caa96-7ece-4f94-b87b-3bf0c5dcb78f None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:16.731719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-6d2caa96-7ece-4f94-b87b-3bf0c5dcb78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:16.848029 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1078/2135] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:53:16.856496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1dc67e-13a3-4b0d-9573-1f9be189da49 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:16.857008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1dc67e-13a3-4b0d-9573-1f9be189da49 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:16.857181 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1dc67e-13a3-4b0d-9573-1f9be189da49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:16.885711 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-5a1dc67e-13a3-4b0d-9573-1f9be189da49 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:16.887514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5a1dc67e-13a3-4b0d-9573-1f9be189da49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:16.906963 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1058/2136] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:53:16.910654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2caa96-7ece-4f94-b87b-3bf0c5dcb78f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:16.926236 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2caa96-7ece-4f94-b87b-3bf0c5dcb78f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:16.932784 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9babe-c953-4766-870a-c5d7e735bda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:16.933493 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9babe-c953-4766-870a-c5d7e735bda5 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:16.933843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9babe-c953-4766-870a-c5d7e735bda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:16.934282 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9babe-c953-4766-870a-c5d7e735bda5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:16.935171 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9babe-c953-4766-870a-c5d7e735bda5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:53:16.935519 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9babe-c953-4766-870a-c5d7e735bda5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:16.937316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9babe-c953-4766-870a-c5d7e735bda5 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:16.952460 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1079/2137] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:53:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:53:16.958956 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1dc67e-13a3-4b0d-9573-1f9be189da49 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:16.969756 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1dc67e-13a3-4b0d-9573-1f9be189da49 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:16.976019 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117a8a0-186a-434e-8f41-b8459aa67bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:16.976660 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117a8a0-186a-434e-8f41-b8459aa67bca admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:16.976938 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117a8a0-186a-434e-8f41-b8459aa67bca admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:16.977200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117a8a0-186a-434e-8f41-b8459aa67bca admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:16.977858 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9117a8a0-186a-434e-8f41-b8459aa67bca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:16.979991 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9117a8a0-186a-434e-8f41-b8459aa67bca admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:17.021868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1059/2138] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:53:16 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:53:17.027412 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9babe-c953-4766-870a-c5d7e735bda5 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:17.039378 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9babe-c953-4766-870a-c5d7e735bda5 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:17.131819 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f2b2e7-76a6-4db1-b864-3a3e59967cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:17.132857 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2b2e7-76a6-4db1-b864-3a3e59967cbe admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:17.133336 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2b2e7-76a6-4db1-b864-3a3e59967cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:17.133825 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2b2e7-76a6-4db1-b864-3a3e59967cbe admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:17.135056 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f2b2e7-76a6-4db1-b864-3a3e59967cbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:17.138663 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f2b2e7-76a6-4db1-b864-3a3e59967cbe admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:17.168256 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1080/2139] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:53:17 2022] POST /identity/v3/users => generated 293 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:53:17.173678 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117a8a0-186a-434e-8f41-b8459aa67bca admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:17.193387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117a8a0-186a-434e-8f41-b8459aa67bca admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:17.204160 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa19ac-4bb9-4c8b-ad19-d48b83049824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:17.204682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa19ac-4bb9-4c8b-ad19-d48b83049824 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:17.205079 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa19ac-4bb9-4c8b-ad19-d48b83049824 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:17.205516 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fa19ac-4bb9-4c8b-ad19-d48b83049824 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:17.206326 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa19ac-4bb9-4c8b-ad19-d48b83049824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:17.208130 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fa19ac-4bb9-4c8b-ad19-d48b83049824 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:17.226590 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1060/2140] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:53:17 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:53:17.232316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f2b2e7-76a6-4db1-b864-3a3e59967cbe admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:17.251361 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f2b2e7-76a6-4db1-b864-3a3e59967cbe admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:17.259216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a53482-2006-4e4c-b865-e9670827cfbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:17.260167 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a53482-2006-4e4c-b865-e9670827cfbe admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:17.260602 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a53482-2006-4e4c-b865-e9670827cfbe admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:17.261153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a53482-2006-4e4c-b865-e9670827cfbe admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:17.262248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a53482-2006-4e4c-b865-e9670827cfbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:17.265169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a53482-2006-4e4c-b865-e9670827cfbe admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:17.285162 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1081/2141] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:53:17 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:53:17.291283 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa19ac-4bb9-4c8b-ad19-d48b83049824 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:17.306797 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fa19ac-4bb9-4c8b-ad19-d48b83049824 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:17.313336 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49cf46-d702-43ba-8b70-747c00f47942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:17.314258 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49cf46-d702-43ba-8b70-747c00f47942 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:17.314756 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49cf46-d702-43ba-8b70-747c00f47942 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:17.315247 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49cf46-d702-43ba-8b70-747c00f47942 admin admin] PATH_INFO: `/v3/projects/f87dce6d7f01455fb8c3cfe128c20e72/users/406be06d6adf40bbb48d9baf12022988/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:17.349895 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49cf46-d702-43ba-8b70-747c00f47942 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f87dce6d7f01455fb8c3cfe128c20e72, user_id=406be06d6adf40bbb48d9baf12022988, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:17.365600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49cf46-d702-43ba-8b70-747c00f47942 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:17.397922 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1061/2142] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:53:17 2022] PUT /identity/v3/projects/f87dce6d7f01455fb8c3cfe128c20e72/users/406be06d6adf40bbb48d9baf12022988/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:53:17.405000 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88f4fb-934b-4409-8935-c6eaa19560fd None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:17.405817 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88f4fb-934b-4409-8935-c6eaa19560fd None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:17.406149 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88f4fb-934b-4409-8935-c6eaa19560fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:17.439636 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-5e88f4fb-934b-4409-8935-c6eaa19560fd None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:17.441658 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-5e88f4fb-934b-4409-8935-c6eaa19560fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:17.537958 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:53:17.575679 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1082/2143] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:53:17.592613 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49cf46-d702-43ba-8b70-747c00f47942 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:17.595619 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6b49cf46-d702-43ba-8b70-747c00f47942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:17.683915 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49cf46-d702-43ba-8b70-747c00f47942 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:17.687672 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba566d6-2127-47a5-82ef-7011d06dc097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:17.688371 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba566d6-2127-47a5-82ef-7011d06dc097 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:17.689009 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba566d6-2127-47a5-82ef-7011d06dc097 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:17.689009 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba566d6-2127-47a5-82ef-7011d06dc097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:17.702269 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba566d6-2127-47a5-82ef-7011d06dc097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:17.703809 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba566d6-2127-47a5-82ef-7011d06dc097 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:17.718125 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1062/2144] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:53:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:53:24.430861 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb83b1-960e-40fb-aeed-825419319476 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:24.431783 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb83b1-960e-40fb-aeed-825419319476 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:24.431783 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb83b1-960e-40fb-aeed-825419319476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:24.469718 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-07eb83b1-960e-40fb-aeed-825419319476 None None] MFA Rules not processed for user `406be06d6adf40bbb48d9baf12022988`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:24.471200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-07eb83b1-960e-40fb-aeed-825419319476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:24.580461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1083/2145] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:53:24.589219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba566d6-2127-47a5-82ef-7011d06dc097 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:24.591302 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-eba566d6-2127-47a5-82ef-7011d06dc097 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:24.728181 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba566d6-2127-47a5-82ef-7011d06dc097 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:24.733943 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d06cc-f483-4417-8ee8-3fd7d5ad457a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:24.734581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d06cc-f483-4417-8ee8-3fd7d5ad457a service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:24.734860 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d06cc-f483-4417-8ee8-3fd7d5ad457a service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:24.735170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d06cc-f483-4417-8ee8-3fd7d5ad457a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:24.749294 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d06cc-f483-4417-8ee8-3fd7d5ad457a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:24.752884 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d06cc-f483-4417-8ee8-3fd7d5ad457a service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:24.764058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1063/2146] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:53:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:53:27.215298 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568814df-364b-418b-901d-a05cc12b7907 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:27.216730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568814df-364b-418b-901d-a05cc12b7907 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:27.217268 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568814df-364b-418b-901d-a05cc12b7907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:27.247690 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-568814df-364b-418b-901d-a05cc12b7907 None None] MFA Rules not processed for user `406be06d6adf40bbb48d9baf12022988`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:27.250723 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-568814df-364b-418b-901d-a05cc12b7907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:27.266188 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1084/2147] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:53:29.132588 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658bb1f-01b9-477b-9047-e74a9c98efad None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:29.133374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658bb1f-01b9-477b-9047-e74a9c98efad None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:29.133374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658bb1f-01b9-477b-9047-e74a9c98efad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:29.158112 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:53:29.168735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-1658bb1f-01b9-477b-9047-e74a9c98efad None None] MFA Rules not processed for user `230efbe75687494fa44777325c8acb4e`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:29.172239 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1658bb1f-01b9-477b-9047-e74a9c98efad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:29.304483 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1064/2148] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:53:33.237055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568814df-364b-418b-901d-a05cc12b7907 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:33.241468 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-568814df-364b-418b-901d-a05cc12b7907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:33.284653 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568814df-364b-418b-901d-a05cc12b7907 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:33.308428 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf7cb3-4127-4ce4-bed9-c20e286ae383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:33.309670 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf7cb3-4127-4ce4-bed9-c20e286ae383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:33.310245 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf7cb3-4127-4ce4-bed9-c20e286ae383 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:33.310852 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf7cb3-4127-4ce4-bed9-c20e286ae383 admin admin] PATH_INFO: `/v3/users/164b1e8ed6c44757b5cd9c2d1accd6ab` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:33.317162 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaf7cb3-4127-4ce4-bed9-c20e286ae383 admin admin] RBAC: Authorizing `identity:delete_user(user_id=164b1e8ed6c44757b5cd9c2d1accd6ab)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:33.327392 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaf7cb3-4127-4ce4-bed9-c20e286ae383 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:33.476198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-edaf7cb3-4127-4ce4-bed9-c20e286ae383 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '164b1e8ed6c44757b5cd9c2d1accd6ab'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:33.505843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-edaf7cb3-4127-4ce4-bed9-c20e286ae383 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '164b1e8ed6c44757b5cd9c2d1accd6ab'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:33.521671 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-edaf7cb3-4127-4ce4-bed9-c20e286ae383 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '164b1e8ed6c44757b5cd9c2d1accd6ab'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:33.539370 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-edaf7cb3-4127-4ce4-bed9-c20e286ae383 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '164b1e8ed6c44757b5cd9c2d1accd6ab'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:33.541040 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-edaf7cb3-4127-4ce4-bed9-c20e286ae383 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '164b1e8ed6c44757b5cd9c2d1accd6ab'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:33.555013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1085/2149] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:53:33 2022] DELETE /identity/v3/users/164b1e8ed6c44757b5cd9c2d1accd6ab => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:53:33.560615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef41cc-0b31-478e-b9f3-a4841848f7bf None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:33.561722 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef41cc-0b31-478e-b9f3-a4841848f7bf None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:33.561964 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef41cc-0b31-478e-b9f3-a4841848f7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:33.620600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-9fef41cc-0b31-478e-b9f3-a4841848f7bf None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:33.623265 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9fef41cc-0b31-478e-b9f3-a4841848f7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:33.779541 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1065/2150] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:53:33.804366 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf7cb3-4127-4ce4-bed9-c20e286ae383 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:33.805990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-edaf7cb3-4127-4ce4-bed9-c20e286ae383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:33.923891 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:53:33.958770 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf7cb3-4127-4ce4-bed9-c20e286ae383 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:33.966601 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697bfd10-f472-4307-ac12-f034bfa953e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:33.967580 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bfd10-f472-4307-ac12-f034bfa953e5 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:33.968115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bfd10-f472-4307-ac12-f034bfa953e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:33.968613 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bfd10-f472-4307-ac12-f034bfa953e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:33.992884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697bfd10-f472-4307-ac12-f034bfa953e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:33.997240 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697bfd10-f472-4307-ac12-f034bfa953e5 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:34.019788 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1086/2151] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:53:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:53:34.279306 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fef41cc-0b31-478e-b9f3-a4841848f7bf None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:34.281892 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9fef41cc-0b31-478e-b9f3-a4841848f7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:34.304361 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fef41cc-0b31-478e-b9f3-a4841848f7bf None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:34.314756 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12247a-e012-4486-a8e0-452169b14290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:34.315928 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12247a-e012-4486-a8e0-452169b14290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:34.316400 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12247a-e012-4486-a8e0-452169b14290 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:34.316863 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12247a-e012-4486-a8e0-452169b14290 admin admin] PATH_INFO: `/v3/users/9af14fae00dc4456aeef1ee27e8a3076` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:34.319976 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12247a-e012-4486-a8e0-452169b14290 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9af14fae00dc4456aeef1ee27e8a3076)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:34.323280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12247a-e012-4486-a8e0-452169b14290 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:34.453021 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-6e12247a-e012-4486-a8e0-452169b14290 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9af14fae00dc4456aeef1ee27e8a3076'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:34.470785 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-6e12247a-e012-4486-a8e0-452169b14290 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9af14fae00dc4456aeef1ee27e8a3076'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:34.489582 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-6e12247a-e012-4486-a8e0-452169b14290 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9af14fae00dc4456aeef1ee27e8a3076'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:34.491501 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-6e12247a-e012-4486-a8e0-452169b14290 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9af14fae00dc4456aeef1ee27e8a3076'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:34.493958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-6e12247a-e012-4486-a8e0-452169b14290 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9af14fae00dc4456aeef1ee27e8a3076'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:34.546958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1066/2152] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:53:34 2022] DELETE /identity/v3/users/9af14fae00dc4456aeef1ee27e8a3076 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:53:34.554254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67454aec-5948-4743-9f60-636aad4b9c6e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:34.556226 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67454aec-5948-4743-9f60-636aad4b9c6e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:34.557071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67454aec-5948-4743-9f60-636aad4b9c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:34.607634 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-67454aec-5948-4743-9f60-636aad4b9c6e None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:34.611088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-67454aec-5948-4743-9f60-636aad4b9c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:34.771717 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1087/2153] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:53:34.833621 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12247a-e012-4486-a8e0-452169b14290 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:34.835454 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6e12247a-e012-4486-a8e0-452169b14290 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:34.902196 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:53:34.959392 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12247a-e012-4486-a8e0-452169b14290 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:34.965201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718f06e7-cd4d-4916-a18d-33153fda18ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:34.967121 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f06e7-cd4d-4916-a18d-33153fda18ea service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:34.967932 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f06e7-cd4d-4916-a18d-33153fda18ea service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:34.968383 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f06e7-cd4d-4916-a18d-33153fda18ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:34.986358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f06e7-cd4d-4916-a18d-33153fda18ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:34.988806 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718f06e7-cd4d-4916-a18d-33153fda18ea service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:35.009274 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1067/2154] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:53:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:53:35.257670 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67454aec-5948-4743-9f60-636aad4b9c6e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:35.261934 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-67454aec-5948-4743-9f60-636aad4b9c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:35.276126 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67454aec-5948-4743-9f60-636aad4b9c6e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:35.282360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc9060-19ab-403c-84ce-283ec9221369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:35.283162 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc9060-19ab-403c-84ce-283ec9221369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:35.283427 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc9060-19ab-403c-84ce-283ec9221369 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:35.283773 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc9060-19ab-403c-84ce-283ec9221369 admin admin] PATH_INFO: `/v3/projects/0785916490864243a6d1e2518481b300` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:35.285086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc9060-19ab-403c-84ce-283ec9221369 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0785916490864243a6d1e2518481b300)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:35.287590 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc9060-19ab-403c-84ce-283ec9221369 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:35.433598 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c2dc9060-19ab-403c-84ce-283ec9221369 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0785916490864243a6d1e2518481b300', 'request_id': 'req-c2dc9060-19ab-403c-84ce-283ec9221369'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:35.449069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c2dc9060-19ab-403c-84ce-283ec9221369 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0785916490864243a6d1e2518481b300', 'request_id': 'req-c2dc9060-19ab-403c-84ce-283ec9221369'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:35.462102 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c2dc9060-19ab-403c-84ce-283ec9221369 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0785916490864243a6d1e2518481b300', 'request_id': 'req-c2dc9060-19ab-403c-84ce-283ec9221369'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:35.478185 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c2dc9060-19ab-403c-84ce-283ec9221369 admin admin] The token cache is being invalidate because project 0785916490864243a6d1e2518481b300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:53:35.478711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c2dc9060-19ab-403c-84ce-283ec9221369 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:35.479834 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c2dc9060-19ab-403c-84ce-283ec9221369 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:35.482075 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1088/2155] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:53:35 2022] DELETE /identity/v3/projects/0785916490864243a6d1e2518481b300 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:53:35.488207 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f0f2f-a1d8-4130-8cf1-c61b69297283 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:35.488596 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f0f2f-a1d8-4130-8cf1-c61b69297283 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:35.491452 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f0f2f-a1d8-4130-8cf1-c61b69297283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:35.513780 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-dc2f0f2f-a1d8-4130-8cf1-c61b69297283 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:35.515823 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-dc2f0f2f-a1d8-4130-8cf1-c61b69297283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:35.659105 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1068/2156] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:53:35.674536 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f3bc0-5ee3-4060-99cb-0126b342f0d6 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:35.675327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f3bc0-5ee3-4060-99cb-0126b342f0d6 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:35.675327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f3bc0-5ee3-4060-99cb-0126b342f0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:35.720037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ed9f3bc0-5ee3-4060-99cb-0126b342f0d6 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:35.736126 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ed9f3bc0-5ee3-4060-99cb-0126b342f0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:35.778272 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1089/2157] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:53:35.780480 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f0f2f-a1d8-4130-8cf1-c61b69297283 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:35.799753 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f0f2f-a1d8-4130-8cf1-c61b69297283 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:35.809787 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a1a1b-82d4-413f-9889-b3100cfc70fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:35.810850 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a1a1b-82d4-413f-9889-b3100cfc70fc admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:35.811701 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a1a1b-82d4-413f-9889-b3100cfc70fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:35.812414 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a1a1b-82d4-413f-9889-b3100cfc70fc admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:35.813562 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a1a1b-82d4-413f-9889-b3100cfc70fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:53:35.814230 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a1a1b-82d4-413f-9889-b3100cfc70fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:35.817135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a1a1b-82d4-413f-9889-b3100cfc70fc admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:35.844883 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1069/2158] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:53:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:53:35.854649 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f3bc0-5ee3-4060-99cb-0126b342f0d6 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:35.872977 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f3bc0-5ee3-4060-99cb-0126b342f0d6 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:35.884429 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ef8f9-2197-4003-ab7c-789f8004ca43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:35.886084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ef8f9-2197-4003-ab7c-789f8004ca43 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:35.886774 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ef8f9-2197-4003-ab7c-789f8004ca43 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:35.887516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ef8f9-2197-4003-ab7c-789f8004ca43 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:35.888895 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ef8f9-2197-4003-ab7c-789f8004ca43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:35.893994 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ef8f9-2197-4003-ab7c-789f8004ca43 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:35.946513 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a1a1b-82d4-413f-9889-b3100cfc70fc admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:35.949227 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d50a1a1b-82d4-413f-9889-b3100cfc70fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:35.959777 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1090/2159] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:53:35 2022] POST /identity/v3/projects => generated 399 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:53:35.964707 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ef8f9-2197-4003-ab7c-789f8004ca43 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:35.969703 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a1a1b-82d4-413f-9889-b3100cfc70fc admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:36.022515 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:53:36.050125 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e597805-cb97-4beb-bcf9-eb39081b8237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:36.050786 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e597805-cb97-4beb-bcf9-eb39081b8237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:36.051320 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e597805-cb97-4beb-bcf9-eb39081b8237 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:36.051596 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e597805-cb97-4beb-bcf9-eb39081b8237 admin admin] PATH_INFO: `/v3/projects/7d68f425160c400ab4ee24a4be3fd9e2` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:36.052801 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e597805-cb97-4beb-bcf9-eb39081b8237 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d68f425160c400ab4ee24a4be3fd9e2)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:36.054831 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e597805-cb97-4beb-bcf9-eb39081b8237 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:36.059522 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ef8f9-2197-4003-ab7c-789f8004ca43 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:36.067136 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec4174-f8ed-4780-861d-ff8311f36eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:36.067914 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec4174-f8ed-4780-861d-ff8311f36eb2 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:36.068375 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec4174-f8ed-4780-861d-ff8311f36eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:36.068677 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec4174-f8ed-4780-861d-ff8311f36eb2 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:36.069595 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ec4174-f8ed-4780-861d-ff8311f36eb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:36.072428 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ec4174-f8ed-4780-861d-ff8311f36eb2 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:36.115073 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1091/2160] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:53:35 2022] POST /identity/v3/users => generated 300 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:53:36.117537 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec4174-f8ed-4780-861d-ff8311f36eb2 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:36.125765 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-8e597805-cb97-4beb-bcf9-eb39081b8237 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d68f425160c400ab4ee24a4be3fd9e2', 'request_id': 'req-8e597805-cb97-4beb-bcf9-eb39081b8237'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:36.131482 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec4174-f8ed-4780-861d-ff8311f36eb2 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:36.136364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-8e597805-cb97-4beb-bcf9-eb39081b8237 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d68f425160c400ab4ee24a4be3fd9e2', 'request_id': 'req-8e597805-cb97-4beb-bcf9-eb39081b8237'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:36.138117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:53:36.146953 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-8e597805-cb97-4beb-bcf9-eb39081b8237 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d68f425160c400ab4ee24a4be3fd9e2', 'request_id': 'req-8e597805-cb97-4beb-bcf9-eb39081b8237'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:36.165172 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-8e597805-cb97-4beb-bcf9-eb39081b8237 admin admin] The token cache is being invalidate because project 7d68f425160c400ab4ee24a4be3fd9e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:53:36.165570 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-8e597805-cb97-4beb-bcf9-eb39081b8237 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:36.166367 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-8e597805-cb97-4beb-bcf9-eb39081b8237 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:36.168084 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1070/2161] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:53:35 2022] DELETE /identity/v3/projects/7d68f425160c400ab4ee24a4be3fd9e2 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:53:36.173495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d522f8-51ba-4252-bab2-06ed3179a39d None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:36.173891 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d522f8-51ba-4252-bab2-06ed3179a39d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:36.174254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d522f8-51ba-4252-bab2-06ed3179a39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:36.202412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-f6d522f8-51ba-4252-bab2-06ed3179a39d None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:36.204432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f6d522f8-51ba-4252-bab2-06ed3179a39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:36.221051 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22955faf-44de-46b6-bd82-38d31b7c6714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:36.221779 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22955faf-44de-46b6-bd82-38d31b7c6714 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:36.222134 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22955faf-44de-46b6-bd82-38d31b7c6714 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:36.222404 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22955faf-44de-46b6-bd82-38d31b7c6714 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:36.223102 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22955faf-44de-46b6-bd82-38d31b7c6714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:36.224756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22955faf-44de-46b6-bd82-38d31b7c6714 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:36.236721 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1092/2162] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 08:53:36.243596 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22955faf-44de-46b6-bd82-38d31b7c6714 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:36.245450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-22955faf-44de-46b6-bd82-38d31b7c6714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:36.260602 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22955faf-44de-46b6-bd82-38d31b7c6714 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:36.264556 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba89b2e-be47-486a-899e-f8304e848582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:36.265162 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba89b2e-be47-486a-899e-f8304e848582 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:36.265433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba89b2e-be47-486a-899e-f8304e848582 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:36.265702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba89b2e-be47-486a-899e-f8304e848582 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:36.266990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba89b2e-be47-486a-899e-f8304e848582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:36.268145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba89b2e-be47-486a-899e-f8304e848582 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:36.280445 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1093/2163] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 08:53:36.285062 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba89b2e-be47-486a-899e-f8304e848582 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:36.293084 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1071/2164] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:53:36.299666 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba89b2e-be47-486a-899e-f8304e848582 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:36.301975 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408f965-6262-4755-9b2b-4314d4637115 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:36.302322 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408f965-6262-4755-9b2b-4314d4637115 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:36.302666 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408f965-6262-4755-9b2b-4314d4637115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:36.308904 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324f8d0-f5f4-40cb-9369-698a77c62347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:36.310201 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324f8d0-f5f4-40cb-9369-698a77c62347 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:36.310642 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324f8d0-f5f4-40cb-9369-698a77c62347 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:36.311477 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324f8d0-f5f4-40cb-9369-698a77c62347 admin admin] PATH_INFO: `/v3/projects/b16215017bc34dc4a3a78d08f4fc7545/users/a16e15f88ff54537aacb9afc495b1964/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:36.331706 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324f8d0-f5f4-40cb-9369-698a77c62347 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b16215017bc34dc4a3a78d08f4fc7545, user_id=a16e15f88ff54537aacb9afc495b1964, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:36.335523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324f8d0-f5f4-40cb-9369-698a77c62347 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:36.343617 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b408f965-6262-4755-9b2b-4314d4637115 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:36.346210 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b408f965-6262-4755-9b2b-4314d4637115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:36.353550 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1094/2165] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:53:36 2022] PUT /identity/v3/projects/b16215017bc34dc4a3a78d08f4fc7545/users/a16e15f88ff54537aacb9afc495b1964/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:53:36.385478 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d3686-dc0e-4a00-9e6b-9e4664191fd5 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:36.386355 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d3686-dc0e-4a00-9e6b-9e4664191fd5 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:36.386847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d3686-dc0e-4a00-9e6b-9e4664191fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:36.454546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-c16d3686-dc0e-4a00-9e6b-9e4664191fd5 None None] MFA Rules not processed for user `a16e15f88ff54537aacb9afc495b1964`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:36.455995 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c16d3686-dc0e-4a00-9e6b-9e4664191fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:36.460241 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1072/2166] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:53:36.470001 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408f965-6262-4755-9b2b-4314d4637115 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:36.482380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408f965-6262-4755-9b2b-4314d4637115 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:36.489307 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39f2ab3-e7b7-4cf4-b86f-985902fec849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:36.490239 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f2ab3-e7b7-4cf4-b86f-985902fec849 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:36.490722 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f2ab3-e7b7-4cf4-b86f-985902fec849 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:36.491167 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f2ab3-e7b7-4cf4-b86f-985902fec849 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:36.492311 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39f2ab3-e7b7-4cf4-b86f-985902fec849 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:53:36.492693 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39f2ab3-e7b7-4cf4-b86f-985902fec849 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:36.494649 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39f2ab3-e7b7-4cf4-b86f-985902fec849 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:36.512776 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1073/2167] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:53:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:53:36.515152 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39f2ab3-e7b7-4cf4-b86f-985902fec849 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:36.527066 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39f2ab3-e7b7-4cf4-b86f-985902fec849 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:36.532864 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1795dd4-9eb0-4f82-9c50-1eafeb3523c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:36.533659 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1795dd4-9eb0-4f82-9c50-1eafeb3523c2 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:36.533948 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1795dd4-9eb0-4f82-9c50-1eafeb3523c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:36.534189 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1795dd4-9eb0-4f82-9c50-1eafeb3523c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:36.534831 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1795dd4-9eb0-4f82-9c50-1eafeb3523c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:36.537483 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1795dd4-9eb0-4f82-9c50-1eafeb3523c2 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:36.576037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1095/2168] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:36 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:53:36.584048 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1074/2169] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:53:36 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:53:36.586681 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d3686-dc0e-4a00-9e6b-9e4664191fd5 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:36.588923 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c16d3686-dc0e-4a00-9e6b-9e4664191fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:36.592948 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1795dd4-9eb0-4f82-9c50-1eafeb3523c2 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:36.629770 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:53:36.665839 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:53:36.680038 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d3686-dc0e-4a00-9e6b-9e4664191fd5 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:36.687344 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1795dd4-9eb0-4f82-9c50-1eafeb3523c2 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:36.697043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41acad-7bc1-4eec-be60-d9ab9b1dcb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:36.698117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41acad-7bc1-4eec-be60-d9ab9b1dcb7e admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:36.698611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41acad-7bc1-4eec-be60-d9ab9b1dcb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:36.699218 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41acad-7bc1-4eec-be60-d9ab9b1dcb7e admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:36.705835 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a41acad-7bc1-4eec-be60-d9ab9b1dcb7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:36.705835 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a41acad-7bc1-4eec-be60-d9ab9b1dcb7e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:36.740953 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1075/2170] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:53:36 2022] POST /identity/v3/users => generated 287 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:53:36.742582 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41acad-7bc1-4eec-be60-d9ab9b1dcb7e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:36.743861 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10411c1-1884-408b-b063-f5b6c2fb718d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:36.744876 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10411c1-1884-408b-b063-f5b6c2fb718d service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:36.745342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10411c1-1884-408b-b063-f5b6c2fb718d service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:36.745801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10411c1-1884-408b-b063-f5b6c2fb718d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:36.755280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41acad-7bc1-4eec-be60-d9ab9b1dcb7e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:36.760314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9ed71-9921-459d-8b80-7885ff6a9160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:36.761027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9ed71-9921-459d-8b80-7885ff6a9160 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:36.761382 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9ed71-9921-459d-8b80-7885ff6a9160 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:36.761723 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9ed71-9921-459d-8b80-7885ff6a9160 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:36.762458 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f9ed71-9921-459d-8b80-7885ff6a9160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:36.764838 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f9ed71-9921-459d-8b80-7885ff6a9160 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:36.788774 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1076/2171] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 08:53:36.793029 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9ed71-9921-459d-8b80-7885ff6a9160 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:36.812853 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9ed71-9921-459d-8b80-7885ff6a9160 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:36.821750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ff412-b53b-4372-8519-c6691ededc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:36.822581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ff412-b53b-4372-8519-c6691ededc9e admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:36.822872 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ff412-b53b-4372-8519-c6691ededc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:36.823297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ff412-b53b-4372-8519-c6691ededc9e admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:36.823845 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ff412-b53b-4372-8519-c6691ededc9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:36.825549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ff412-b53b-4372-8519-c6691ededc9e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:36.831413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10411c1-1884-408b-b063-f5b6c2fb718d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:36.833267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10411c1-1884-408b-b063-f5b6c2fb718d service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:36.839556 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1077/2172] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 08:53:36.844484 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ff412-b53b-4372-8519-c6691ededc9e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:36.850101 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1096/2173] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:53:36 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:53:36.855585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ff412-b53b-4372-8519-c6691ededc9e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:36.862018 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f4bf8-5e7c-446f-9536-1ff9e7dcdd6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:36.862926 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f4bf8-5e7c-446f-9536-1ff9e7dcdd6b admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:36.863191 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f4bf8-5e7c-446f-9536-1ff9e7dcdd6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:36.863468 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7f4bf8-5e7c-446f-9536-1ff9e7dcdd6b admin admin] PATH_INFO: `/v3/projects/9d3a242b11864190bd3f3f7e3a748111/users/32b3dea804584d78a659a6bb019bc66f/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:36.885693 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7f4bf8-5e7c-446f-9536-1ff9e7dcdd6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d3a242b11864190bd3f3f7e3a748111, user_id=32b3dea804584d78a659a6bb019bc66f, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:36.890918 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7f4bf8-5e7c-446f-9536-1ff9e7dcdd6b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:36.927903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1078/2174] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:53:36 2022] PUT /identity/v3/projects/9d3a242b11864190bd3f3f7e3a748111/users/32b3dea804584d78a659a6bb019bc66f/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:53:36.948588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091bb02-0ecd-45a0-9676-fb6d4c90bfd3 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:36.949274 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091bb02-0ecd-45a0-9676-fb6d4c90bfd3 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:36.949829 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091bb02-0ecd-45a0-9676-fb6d4c90bfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:36.997524 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-2091bb02-0ecd-45a0-9676-fb6d4c90bfd3 None None] MFA Rules not processed for user `32b3dea804584d78a659a6bb019bc66f`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:36.999619 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2091bb02-0ecd-45a0-9676-fb6d4c90bfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:37.102333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1097/2175] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:53:37.117550 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f4bf8-5e7c-446f-9536-1ff9e7dcdd6b admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:37.179222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7f4bf8-5e7c-446f-9536-1ff9e7dcdd6b admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:37.183368 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649f355-c4f1-43d5-94ba-238d7dcfad4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:37.183989 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649f355-c4f1-43d5-94ba-238d7dcfad4a service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:37.184267 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649f355-c4f1-43d5-94ba-238d7dcfad4a service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:37.184538 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649f355-c4f1-43d5-94ba-238d7dcfad4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:37.194332 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649f355-c4f1-43d5-94ba-238d7dcfad4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:37.195916 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649f355-c4f1-43d5-94ba-238d7dcfad4a service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:37.208022 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1079/2176] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:53:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:53:42.599790 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091bb02-0ecd-45a0-9676-fb6d4c90bfd3 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:42.633561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:53:42.692413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091bb02-0ecd-45a0-9676-fb6d4c90bfd3 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:42.697402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb901f84-b7ec-425d-8352-47707f812bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:42.698083 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb901f84-b7ec-425d-8352-47707f812bdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:42.698357 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb901f84-b7ec-425d-8352-47707f812bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:42.698623 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb901f84-b7ec-425d-8352-47707f812bdb admin admin] PATH_INFO: `/v3/users/32b3dea804584d78a659a6bb019bc66f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:42.699872 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb901f84-b7ec-425d-8352-47707f812bdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=32b3dea804584d78a659a6bb019bc66f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:42.701995 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb901f84-b7ec-425d-8352-47707f812bdb admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:42.760216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-fb901f84-b7ec-425d-8352-47707f812bdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32b3dea804584d78a659a6bb019bc66f'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:42.776777 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-fb901f84-b7ec-425d-8352-47707f812bdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32b3dea804584d78a659a6bb019bc66f'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:42.788116 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-fb901f84-b7ec-425d-8352-47707f812bdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32b3dea804584d78a659a6bb019bc66f'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:42.800558 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-fb901f84-b7ec-425d-8352-47707f812bdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32b3dea804584d78a659a6bb019bc66f'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:42.801767 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-fb901f84-b7ec-425d-8352-47707f812bdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32b3dea804584d78a659a6bb019bc66f'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:42.811783 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1098/2177] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:53:42 2022] DELETE /identity/v3/users/32b3dea804584d78a659a6bb019bc66f => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:53:42.821407 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1d55a-c7be-43c4-b135-68276b88b63c None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:42.822242 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1d55a-c7be-43c4-b135-68276b88b63c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:42.823277 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1d55a-c7be-43c4-b135-68276b88b63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:42.847125 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-20c1d55a-c7be-43c4-b135-68276b88b63c None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:42.849167 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-20c1d55a-c7be-43c4-b135-68276b88b63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:42.974947 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1080/2178] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:53:42.990403 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb901f84-b7ec-425d-8352-47707f812bdb admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:42.993504 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-fb901f84-b7ec-425d-8352-47707f812bdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:43.107484 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb901f84-b7ec-425d-8352-47707f812bdb admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:43.114599 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f43cb-3c00-4b5f-b748-0871aa7b45f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:43.116036 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f43cb-3c00-4b5f-b748-0871aa7b45f8 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:43.116036 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f43cb-3c00-4b5f-b748-0871aa7b45f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:43.116517 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f43cb-3c00-4b5f-b748-0871aa7b45f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:43.137200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f43cb-3c00-4b5f-b748-0871aa7b45f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:43.141295 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f43cb-3c00-4b5f-b748-0871aa7b45f8 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:43.163288 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1099/2179] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:53:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:53:43.610511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1d55a-c7be-43c4-b135-68276b88b63c None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:43.612338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-20c1d55a-c7be-43c4-b135-68276b88b63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:43.626377 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1d55a-c7be-43c4-b135-68276b88b63c None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:43.631871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25888001-7888-4c44-8974-35c8312a78e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:43.632552 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25888001-7888-4c44-8974-35c8312a78e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:43.632826 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25888001-7888-4c44-8974-35c8312a78e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:43.633085 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25888001-7888-4c44-8974-35c8312a78e3 admin admin] PATH_INFO: `/v3/projects/9d3a242b11864190bd3f3f7e3a748111` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:43.634351 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25888001-7888-4c44-8974-35c8312a78e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d3a242b11864190bd3f3f7e3a748111)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:43.636410 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25888001-7888-4c44-8974-35c8312a78e3 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:43.680649 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:53:43.704824 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-25888001-7888-4c44-8974-35c8312a78e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d3a242b11864190bd3f3f7e3a748111', 'request_id': 'req-25888001-7888-4c44-8974-35c8312a78e3'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:43.719532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-25888001-7888-4c44-8974-35c8312a78e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d3a242b11864190bd3f3f7e3a748111', 'request_id': 'req-25888001-7888-4c44-8974-35c8312a78e3'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:43.729283 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-25888001-7888-4c44-8974-35c8312a78e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d3a242b11864190bd3f3f7e3a748111', 'request_id': 'req-25888001-7888-4c44-8974-35c8312a78e3'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:43.747981 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-25888001-7888-4c44-8974-35c8312a78e3 admin admin] The token cache is being invalidate because project 9d3a242b11864190bd3f3f7e3a748111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:53:43.749119 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-25888001-7888-4c44-8974-35c8312a78e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:43.749582 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-25888001-7888-4c44-8974-35c8312a78e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:43.751999 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1081/2180] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:53:43 2022] DELETE /identity/v3/projects/9d3a242b11864190bd3f3f7e3a748111 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:53:43.757136 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41bebe-152a-48e9-b152-242a72aee396 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:43.757735 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41bebe-152a-48e9-b152-242a72aee396 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:43.757848 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41bebe-152a-48e9-b152-242a72aee396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:43.782559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-1c41bebe-152a-48e9-b152-242a72aee396 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:43.783883 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1c41bebe-152a-48e9-b152-242a72aee396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:43.855478 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:53:43.888828 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1100/2181] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:53:43.900661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe186a-5a18-4271-962a-55b73b8ac400 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:43.901552 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe186a-5a18-4271-962a-55b73b8ac400 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:43.901979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe186a-5a18-4271-962a-55b73b8ac400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:43.928376 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-d5fe186a-5a18-4271-962a-55b73b8ac400 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:43.929637 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d5fe186a-5a18-4271-962a-55b73b8ac400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:43.938966 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1082/2182] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:53:43.943567 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41bebe-152a-48e9-b152-242a72aee396 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:43.954815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41bebe-152a-48e9-b152-242a72aee396 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:43.959784 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5847ecc-22e6-45b5-8e73-c95180bfbd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:43.960763 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5847ecc-22e6-45b5-8e73-c95180bfbd4d admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:43.960763 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5847ecc-22e6-45b5-8e73-c95180bfbd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:43.960969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5847ecc-22e6-45b5-8e73-c95180bfbd4d admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:43.961615 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5847ecc-22e6-45b5-8e73-c95180bfbd4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:53:43.961942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5847ecc-22e6-45b5-8e73-c95180bfbd4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:43.963667 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5847ecc-22e6-45b5-8e73-c95180bfbd4d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:43.985076 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1101/2183] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:53:43.985980 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe186a-5a18-4271-962a-55b73b8ac400 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:44.002055 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe186a-5a18-4271-962a-55b73b8ac400 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:44.012159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef43fa-0cad-4d10-b36e-422b82ec133a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:44.013118 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef43fa-0cad-4d10-b36e-422b82ec133a admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:44.013554 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef43fa-0cad-4d10-b36e-422b82ec133a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:44.013994 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef43fa-0cad-4d10-b36e-422b82ec133a admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:44.014873 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef43fa-0cad-4d10-b36e-422b82ec133a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:44.017309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef43fa-0cad-4d10-b36e-422b82ec133a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:44.060977 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1083/2184] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:53:43 2022] POST /identity/v3/projects => generated 363 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:53:44.067354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5847ecc-22e6-45b5-8e73-c95180bfbd4d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:44.100930 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5847ecc-22e6-45b5-8e73-c95180bfbd4d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:44.261427 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2c6b5-461d-449b-925e-8101248c6629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:44.262073 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2c6b5-461d-449b-925e-8101248c6629 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:44.262386 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2c6b5-461d-449b-925e-8101248c6629 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:44.262604 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2c6b5-461d-449b-925e-8101248c6629 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:44.263887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2c6b5-461d-449b-925e-8101248c6629 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:44.266218 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2c6b5-461d-449b-925e-8101248c6629 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:44.306312 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1102/2185] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:53:44 2022] POST /identity/v3/users => generated 282 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:53:44.310630 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef43fa-0cad-4d10-b36e-422b82ec133a admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:44.348171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef43fa-0cad-4d10-b36e-422b82ec133a admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:44.362991 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde88d2-229e-4fa1-a8fc-b336c1dad558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:44.364141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde88d2-229e-4fa1-a8fc-b336c1dad558 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:44.366149 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde88d2-229e-4fa1-a8fc-b336c1dad558 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:44.366762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde88d2-229e-4fa1-a8fc-b336c1dad558 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:44.368834 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde88d2-229e-4fa1-a8fc-b336c1dad558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:44.370674 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde88d2-229e-4fa1-a8fc-b336c1dad558 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:44.428030 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1084/2186] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:53:44.428263 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2c6b5-461d-449b-925e-8101248c6629 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:44.438425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2c6b5-461d-449b-925e-8101248c6629 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:44.447129 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08da4e-8844-469e-8a36-7539ba4a0db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:44.448339 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08da4e-8844-469e-8a36-7539ba4a0db8 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:44.448490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08da4e-8844-469e-8a36-7539ba4a0db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:44.448897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08da4e-8844-469e-8a36-7539ba4a0db8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:44.449762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08da4e-8844-469e-8a36-7539ba4a0db8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:44.452107 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08da4e-8844-469e-8a36-7539ba4a0db8 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:44.469076 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1103/2187] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:53:44.475008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde88d2-229e-4fa1-a8fc-b336c1dad558 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:44.490864 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde88d2-229e-4fa1-a8fc-b336c1dad558 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:44.496804 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdab133-5bcd-4b5a-a5a2-af48824c10dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:44.497661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdab133-5bcd-4b5a-a5a2-af48824c10dc admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:44.498817 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdab133-5bcd-4b5a-a5a2-af48824c10dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:44.500094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdab133-5bcd-4b5a-a5a2-af48824c10dc admin admin] PATH_INFO: `/v3/projects/30fad5203b0147acb083e51ed8eed9bf/users/100dde650f224b6f9654d7c8bb36d367/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:44.530308 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdab133-5bcd-4b5a-a5a2-af48824c10dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=30fad5203b0147acb083e51ed8eed9bf, user_id=100dde650f224b6f9654d7c8bb36d367, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:44.533201 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdab133-5bcd-4b5a-a5a2-af48824c10dc admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:44.564823 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1085/2188] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:53:44 2022] PUT /identity/v3/projects/30fad5203b0147acb083e51ed8eed9bf/users/100dde650f224b6f9654d7c8bb36d367/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:53:44.568768 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fec805-5d2c-41ef-bed3-047681a74e92 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:44.569243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fec805-5d2c-41ef-bed3-047681a74e92 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:44.569825 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fec805-5d2c-41ef-bed3-047681a74e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:44.665679 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-66fec805-5d2c-41ef-bed3-047681a74e92 None None] MFA Rules not processed for user `100dde650f224b6f9654d7c8bb36d367`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:44.671007 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-66fec805-5d2c-41ef-bed3-047681a74e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:44.853197 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1104/2189] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:53:44.863096 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdab133-5bcd-4b5a-a5a2-af48824c10dc admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:44.865668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5cdab133-5bcd-4b5a-a5a2-af48824c10dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:44.941609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdab133-5bcd-4b5a-a5a2-af48824c10dc admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:44.945518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ecc19-a93a-468f-9927-b46d2495cd8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:44.946194 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ecc19-a93a-468f-9927-b46d2495cd8e service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:44.946442 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ecc19-a93a-468f-9927-b46d2495cd8e service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:44.946710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ecc19-a93a-468f-9927-b46d2495cd8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:44.958423 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3ecc19-a93a-468f-9927-b46d2495cd8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:44.960020 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3ecc19-a93a-468f-9927-b46d2495cd8e service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:44.973732 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1086/2190] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:53:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:53:45.397136 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fec805-5d2c-41ef-bed3-047681a74e92 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:45.399450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-66fec805-5d2c-41ef-bed3-047681a74e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:45.460014 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:53:45.490384 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fec805-5d2c-41ef-bed3-047681a74e92 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:45.502182 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce7944-274e-413c-b917-7828a444d234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:45.502752 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce7944-274e-413c-b917-7828a444d234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:45.503496 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce7944-274e-413c-b917-7828a444d234 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:45.503496 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce7944-274e-413c-b917-7828a444d234 admin admin] PATH_INFO: `/v3/users/a16e15f88ff54537aacb9afc495b1964` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:45.504929 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce7944-274e-413c-b917-7828a444d234 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a16e15f88ff54537aacb9afc495b1964)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:45.508762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce7944-274e-413c-b917-7828a444d234 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:45.581635 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-48ce7944-274e-413c-b917-7828a444d234 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a16e15f88ff54537aacb9afc495b1964'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:45.606308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-48ce7944-274e-413c-b917-7828a444d234 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a16e15f88ff54537aacb9afc495b1964'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:45.621297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-48ce7944-274e-413c-b917-7828a444d234 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a16e15f88ff54537aacb9afc495b1964'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:45.639667 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-48ce7944-274e-413c-b917-7828a444d234 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a16e15f88ff54537aacb9afc495b1964'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:45.640899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-48ce7944-274e-413c-b917-7828a444d234 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a16e15f88ff54537aacb9afc495b1964'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:45.649972 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1105/2191] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:53:45 2022] DELETE /identity/v3/users/a16e15f88ff54537aacb9afc495b1964 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:53:45.655750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d5046-f477-426e-b8bb-0cde588d0ec3 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:45.656033 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d5046-f477-426e-b8bb-0cde588d0ec3 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:45.656439 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d5046-f477-426e-b8bb-0cde588d0ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:45.682057 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-744d5046-f477-426e-b8bb-0cde588d0ec3 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:45.683315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-744d5046-f477-426e-b8bb-0cde588d0ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:45.753354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:53:45.766117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1087/2192] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:53:45.782776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce7944-274e-413c-b917-7828a444d234 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:45.785053 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-48ce7944-274e-413c-b917-7828a444d234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:45.871612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce7944-274e-413c-b917-7828a444d234 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:45.876939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05140d-44b5-4bd3-87ac-0b74e2e9610d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:45.877768 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05140d-44b5-4bd3-87ac-0b74e2e9610d service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:45.877768 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05140d-44b5-4bd3-87ac-0b74e2e9610d service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:45.878642 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05140d-44b5-4bd3-87ac-0b74e2e9610d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:45.889411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05140d-44b5-4bd3-87ac-0b74e2e9610d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:45.891076 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05140d-44b5-4bd3-87ac-0b74e2e9610d service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:45.902609 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1106/2193] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:53:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:53:46.529658 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d5046-f477-426e-b8bb-0cde588d0ec3 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:46.532674 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-744d5046-f477-426e-b8bb-0cde588d0ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:46.551394 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d5046-f477-426e-b8bb-0cde588d0ec3 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:46.556721 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf605194-e076-41d4-9167-b183d9817fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:46.557490 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf605194-e076-41d4-9167-b183d9817fb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:46.557856 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf605194-e076-41d4-9167-b183d9817fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:46.558253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf605194-e076-41d4-9167-b183d9817fb3 admin admin] PATH_INFO: `/v3/projects/b16215017bc34dc4a3a78d08f4fc7545` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:46.559722 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf605194-e076-41d4-9167-b183d9817fb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b16215017bc34dc4a3a78d08f4fc7545)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:46.562505 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf605194-e076-41d4-9167-b183d9817fb3 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:46.634964 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-bf605194-e076-41d4-9167-b183d9817fb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b16215017bc34dc4a3a78d08f4fc7545', 'request_id': 'req-bf605194-e076-41d4-9167-b183d9817fb3'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:46.650507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-bf605194-e076-41d4-9167-b183d9817fb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b16215017bc34dc4a3a78d08f4fc7545', 'request_id': 'req-bf605194-e076-41d4-9167-b183d9817fb3'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:46.684916 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-bf605194-e076-41d4-9167-b183d9817fb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b16215017bc34dc4a3a78d08f4fc7545', 'request_id': 'req-bf605194-e076-41d4-9167-b183d9817fb3'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:46.713746 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-bf605194-e076-41d4-9167-b183d9817fb3 admin admin] The token cache is being invalidate because project b16215017bc34dc4a3a78d08f4fc7545 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:53:46.714290 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-bf605194-e076-41d4-9167-b183d9817fb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:46.715588 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-bf605194-e076-41d4-9167-b183d9817fb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:53:46.718086 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1088/2194] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:53:46 2022] DELETE /identity/v3/projects/b16215017bc34dc4a3a78d08f4fc7545 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:53:46.726691 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835bab0-3a0c-4322-a1a1-b147aada4acf None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:46.726691 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835bab0-3a0c-4322-a1a1-b147aada4acf None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:46.727776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835bab0-3a0c-4322-a1a1-b147aada4acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:46.759549 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-a835bab0-3a0c-4322-a1a1-b147aada4acf None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:46.761445 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a835bab0-3a0c-4322-a1a1-b147aada4acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:46.882597 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:53:46.918363 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1107/2195] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:53:46.930357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902dd02-33b3-48b2-80bc-104f19e66737 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:46.931247 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902dd02-33b3-48b2-80bc-104f19e66737 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:46.931712 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902dd02-33b3-48b2-80bc-104f19e66737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:46.964143 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-3902dd02-33b3-48b2-80bc-104f19e66737 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:46.966167 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3902dd02-33b3-48b2-80bc-104f19e66737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:46.975780 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1089/2196] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:53:46.980198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a835bab0-3a0c-4322-a1a1-b147aada4acf None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:47.005478 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a835bab0-3a0c-4322-a1a1-b147aada4acf None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:47.018017 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59237417-67e7-485b-b635-2022203e951b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:47.019914 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59237417-67e7-485b-b635-2022203e951b admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:47.021051 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59237417-67e7-485b-b635-2022203e951b admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:47.021723 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59237417-67e7-485b-b635-2022203e951b admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:47.022818 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59237417-67e7-485b-b635-2022203e951b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:53:47.023277 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59237417-67e7-485b-b635-2022203e951b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:47.026445 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59237417-67e7-485b-b635-2022203e951b admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:47.059603 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1108/2197] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:53:47.061301 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902dd02-33b3-48b2-80bc-104f19e66737 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:47.080937 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902dd02-33b3-48b2-80bc-104f19e66737 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:47.089879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c43fa-bc4e-4c44-be33-cef420715b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:47.090668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c43fa-bc4e-4c44-be33-cef420715b59 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:47.090941 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c43fa-bc4e-4c44-be33-cef420715b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:47.091219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c43fa-bc4e-4c44-be33-cef420715b59 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:47.091938 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c43fa-bc4e-4c44-be33-cef420715b59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:47.094675 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c43fa-bc4e-4c44-be33-cef420715b59 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:47.135571 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1090/2198] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:53:47 2022] POST /identity/v3/projects => generated 373 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:53:47.141089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59237417-67e7-485b-b635-2022203e951b admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:47.169838 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59237417-67e7-485b-b635-2022203e951b admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:47.257380 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be2c3c-1e89-43d9-b666-ac5badf3a192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:47.258326 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be2c3c-1e89-43d9-b666-ac5badf3a192 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:47.259215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be2c3c-1e89-43d9-b666-ac5badf3a192 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:47.259691 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be2c3c-1e89-43d9-b666-ac5badf3a192 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:47.260536 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be2c3c-1e89-43d9-b666-ac5badf3a192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:47.263466 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be2c3c-1e89-43d9-b666-ac5badf3a192 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:47.294198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1109/2199] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:53:47 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:53:47.299222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c43fa-bc4e-4c44-be33-cef420715b59 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:47.312379 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c43fa-bc4e-4c44-be33-cef420715b59 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:47.319358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce3ae5-fddf-4365-b7f6-4ca1827e4236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:47.320074 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce3ae5-fddf-4365-b7f6-4ca1827e4236 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:47.320629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce3ae5-fddf-4365-b7f6-4ca1827e4236 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:47.320916 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce3ae5-fddf-4365-b7f6-4ca1827e4236 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:47.321626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdce3ae5-fddf-4365-b7f6-4ca1827e4236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:47.323424 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdce3ae5-fddf-4365-b7f6-4ca1827e4236 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:47.336607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1091/2200] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:53:47.341217 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be2c3c-1e89-43d9-b666-ac5badf3a192 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:47.356331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be2c3c-1e89-43d9-b666-ac5badf3a192 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:47.363684 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7842bd-45ee-47a4-a40a-a6751c012311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:47.364916 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7842bd-45ee-47a4-a40a-a6751c012311 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:47.364916 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7842bd-45ee-47a4-a40a-a6751c012311 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:47.365330 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7842bd-45ee-47a4-a40a-a6751c012311 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:47.366496 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7842bd-45ee-47a4-a40a-a6751c012311 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:47.366496 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7842bd-45ee-47a4-a40a-a6751c012311 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:47.387000 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1110/2201] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:53:47.392523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce3ae5-fddf-4365-b7f6-4ca1827e4236 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:47.405153 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce3ae5-fddf-4365-b7f6-4ca1827e4236 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:47.410533 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2b2f1-c8bf-4728-b4eb-ee1207e629f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:47.411275 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2b2f1-c8bf-4728-b4eb-ee1207e629f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:47.411555 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2b2f1-c8bf-4728-b4eb-ee1207e629f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:47.411833 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2b2f1-c8bf-4728-b4eb-ee1207e629f5 admin admin] PATH_INFO: `/v3/projects/37cc63bca9274f9ba58f8a92a499a36d/users/229fe8f73e074ef89b1110da844268df/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:47.432543 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2b2f1-c8bf-4728-b4eb-ee1207e629f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37cc63bca9274f9ba58f8a92a499a36d, user_id=229fe8f73e074ef89b1110da844268df, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:47.436438 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2b2f1-c8bf-4728-b4eb-ee1207e629f5 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:47.461562 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1092/2202] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:53:47 2022] PUT /identity/v3/projects/37cc63bca9274f9ba58f8a92a499a36d/users/229fe8f73e074ef89b1110da844268df/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:53:47.470622 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aacfbc-79c5-4a70-b29b-bac4e0b020c2 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:47.470987 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aacfbc-79c5-4a70-b29b-bac4e0b020c2 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:47.471507 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aacfbc-79c5-4a70-b29b-bac4e0b020c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:47.507975 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-29aacfbc-79c5-4a70-b29b-bac4e0b020c2 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:47.510804 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-29aacfbc-79c5-4a70-b29b-bac4e0b020c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:47.643002 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1111/2203] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:53:47.659868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2b2f1-c8bf-4728-b4eb-ee1207e629f5 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:47.665204 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b8a2b2f1-c8bf-4728-b4eb-ee1207e629f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:47.740789 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2b2f1-c8bf-4728-b4eb-ee1207e629f5 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:47.745948 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3741a96d-3132-455f-b5f4-34f4199a9d06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:47.747820 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741a96d-3132-455f-b5f4-34f4199a9d06 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:47.748350 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741a96d-3132-455f-b5f4-34f4199a9d06 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:47.748904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3741a96d-3132-455f-b5f4-34f4199a9d06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:47.761918 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3741a96d-3132-455f-b5f4-34f4199a9d06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:47.763590 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3741a96d-3132-455f-b5f4-34f4199a9d06 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:47.775663 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1093/2204] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:53:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:53:54.891611 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ff4e3-7bcf-4885-82b8-e95b116cf649 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:54.892453 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ff4e3-7bcf-4885-82b8-e95b116cf649 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:54.892453 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ff4e3-7bcf-4885-82b8-e95b116cf649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:54.926395 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-188ff4e3-7bcf-4885-82b8-e95b116cf649 None None] MFA Rules not processed for user `229fe8f73e074ef89b1110da844268df`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:53:54.927692 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-188ff4e3-7bcf-4885-82b8-e95b116cf649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:53:55.002163 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:53:55.014326 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1112/2205] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:53:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:53:55.020021 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3741a96d-3132-455f-b5f4-34f4199a9d06 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:55.035064 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3741a96d-3132-455f-b5f4-34f4199a9d06 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:55.040075 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42940da-340a-4b9f-9882-6953671fedbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:55.040708 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42940da-340a-4b9f-9882-6953671fedbb admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:55.040996 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42940da-340a-4b9f-9882-6953671fedbb admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:55.041239 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42940da-340a-4b9f-9882-6953671fedbb admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:55.041926 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42940da-340a-4b9f-9882-6953671fedbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:55.043948 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42940da-340a-4b9f-9882-6953671fedbb admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:55.071032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:53:55.079746 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1094/2206] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:53:55 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:53:55.086105 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188ff4e3-7bcf-4885-82b8-e95b116cf649 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:55.101880 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188ff4e3-7bcf-4885-82b8-e95b116cf649 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:55.180358 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9ea0cf-9843-429d-a797-33541d2507b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:55.180988 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ea0cf-9843-429d-a797-33541d2507b8 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:55.181264 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ea0cf-9843-429d-a797-33541d2507b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:55.181542 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ea0cf-9843-429d-a797-33541d2507b8 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:55.182198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9ea0cf-9843-429d-a797-33541d2507b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:55.184450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9ea0cf-9843-429d-a797-33541d2507b8 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:55.216471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1113/2207] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:53:55 2022] POST /identity/v3/users => generated 293 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:53:55.219376 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42940da-340a-4b9f-9882-6953671fedbb admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:55.235564 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42940da-340a-4b9f-9882-6953671fedbb admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:55.243679 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ee0f7-a453-4be3-9abe-712de8d89e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:55.244420 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ee0f7-a453-4be3-9abe-712de8d89e81 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:55.244765 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ee0f7-a453-4be3-9abe-712de8d89e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:55.245101 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ee0f7-a453-4be3-9abe-712de8d89e81 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:55.245807 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ee0f7-a453-4be3-9abe-712de8d89e81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:55.247759 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ee0f7-a453-4be3-9abe-712de8d89e81 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:55.261377 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1095/2208] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:53:55.266921 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9ea0cf-9843-429d-a797-33541d2507b8 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:55.278245 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9ea0cf-9843-429d-a797-33541d2507b8 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:55.285505 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f758a-bb97-4b04-b588-a0a1b61d7582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:55.286140 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f758a-bb97-4b04-b588-a0a1b61d7582 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:55.286455 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f758a-bb97-4b04-b588-a0a1b61d7582 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:55.287268 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f758a-bb97-4b04-b588-a0a1b61d7582 admin admin] PATH_INFO: `/v3/projects/37a83a062eee4e989da032ca1857a68d/users/d4460301d0e94eeca47cd87e4d04299a/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:55.307150 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84f758a-bb97-4b04-b588-a0a1b61d7582 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37a83a062eee4e989da032ca1857a68d, user_id=d4460301d0e94eeca47cd87e4d04299a, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:55.310070 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84f758a-bb97-4b04-b588-a0a1b61d7582 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:55.332401 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1114/2209] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:53:55 2022] PUT /identity/v3/projects/37a83a062eee4e989da032ca1857a68d/users/d4460301d0e94eeca47cd87e4d04299a/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:53:55.337779 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ee0f7-a453-4be3-9abe-712de8d89e81 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:55.356400 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ee0f7-a453-4be3-9abe-712de8d89e81 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:55.447599 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91545e1a-8b8a-4c77-93d6-cf6c9a0bfb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:55.448647 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91545e1a-8b8a-4c77-93d6-cf6c9a0bfb1a admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:55.449121 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91545e1a-8b8a-4c77-93d6-cf6c9a0bfb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:55.449593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91545e1a-8b8a-4c77-93d6-cf6c9a0bfb1a admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:55.450740 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91545e1a-8b8a-4c77-93d6-cf6c9a0bfb1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:55.454223 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91545e1a-8b8a-4c77-93d6-cf6c9a0bfb1a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:55.476785 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1096/2210] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:53:55.481452 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f758a-bb97-4b04-b588-a0a1b61d7582 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:53:55.494484 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f758a-bb97-4b04-b588-a0a1b61d7582 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:53:55.499739 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6b1da-7562-4dfb-b16b-05fa4f800fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:53:55.500452 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6b1da-7562-4dfb-b16b-05fa4f800fd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:53:55.500727 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6b1da-7562-4dfb-b16b-05fa4f800fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:53:55.500999 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6b1da-7562-4dfb-b16b-05fa4f800fd3 admin admin] PATH_INFO: `/v3/projects/37a83a062eee4e989da032ca1857a68d/users/d4460301d0e94eeca47cd87e4d04299a/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:53:55.503135 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae6b1da-7562-4dfb-b16b-05fa4f800fd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37a83a062eee4e989da032ca1857a68d, user_id=d4460301d0e94eeca47cd87e4d04299a, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:53:55.506020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae6b1da-7562-4dfb-b16b-05fa4f800fd3 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:53:55.518473 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1115/2211] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:53:55 2022] PUT /identity/v3/projects/37a83a062eee4e989da032ca1857a68d/users/d4460301d0e94eeca47cd87e4d04299a/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:54:02.650312 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975befc-b9fe-419b-8541-9a3bb9c5c768 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:54:02.650969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975befc-b9fe-419b-8541-9a3bb9c5c768 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:54:02.651102 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975befc-b9fe-419b-8541-9a3bb9c5c768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:54:02.702547 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b975befc-b9fe-419b-8541-9a3bb9c5c768 None None] MFA Rules not processed for user `d4460301d0e94eeca47cd87e4d04299a`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:54:02.704657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b975befc-b9fe-419b-8541-9a3bb9c5c768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:54:02.832765 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1097/2212] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:54:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:54:02.842393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6b1da-7562-4dfb-b16b-05fa4f800fd3 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:54:02.864325 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1ae6b1da-7562-4dfb-b16b-05fa4f800fd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:54:02.988206 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6b1da-7562-4dfb-b16b-05fa4f800fd3 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:54:02.996702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847b68f-7088-4e8c-949e-25955b060e3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:54:02.997535 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847b68f-7088-4e8c-949e-25955b060e3e service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:54:02.997893 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847b68f-7088-4e8c-949e-25955b060e3e service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:54:02.998167 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847b68f-7088-4e8c-949e-25955b060e3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:54:03.070085 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:54:03.088400 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0847b68f-7088-4e8c-949e-25955b060e3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:54:03.090711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0847b68f-7088-4e8c-949e-25955b060e3e service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:54:03.102499 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1116/2213] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:54:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:54:04.551357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65daec81-cd1d-49a8-958a-812166105b81 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:54:04.552184 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65daec81-cd1d-49a8-958a-812166105b81 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:54:04.552184 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65daec81-cd1d-49a8-958a-812166105b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:54:04.586292 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-65daec81-cd1d-49a8-958a-812166105b81 None None] MFA Rules not processed for user `229fe8f73e074ef89b1110da844268df`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:54:04.587833 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-65daec81-cd1d-49a8-958a-812166105b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:54:04.597525 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1098/2214] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:54:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:54:13.097766 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=7 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:54:14.574943 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=71 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:54:29.763213 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6fc73-7104-4ae1-bbe2-03fb9b7db261 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:54:29.763711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6fc73-7104-4ae1-bbe2-03fb9b7db261 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:54:29.763800 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6fc73-7104-4ae1-bbe2-03fb9b7db261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:54:29.787475 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-60d6fc73-7104-4ae1-bbe2-03fb9b7db261 None None] MFA Rules not processed for user `df07cffd82c74621855c8b8d7bfa989d`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:54:29.788726 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-60d6fc73-7104-4ae1-bbe2-03fb9b7db261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:54:29.861957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1117/2215] 10.222.0.71 () {58 vars in 1009 bytes} [Fri Aug 26 08:54:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:54:29.913232 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65daec81-cd1d-49a8-958a-812166105b81 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:54:29.915063 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-65daec81-cd1d-49a8-958a-812166105b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:54:30.004957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65daec81-cd1d-49a8-958a-812166105b81 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:54:30.012710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60eb20d-b0a2-42d4-9e31-7db405cc5544 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1723e961375b45b7b3bd99ddeb69683f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:54:30.013389 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60eb20d-b0a2-42d4-9e31-7db405cc5544 service swift] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:54:30.013772 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60eb20d-b0a2-42d4-9e31-7db405cc5544 service swift] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:54:30.014061 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60eb20d-b0a2-42d4-9e31-7db405cc5544 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:54:30.026877 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60eb20d-b0a2-42d4-9e31-7db405cc5544 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:54:30.028588 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60eb20d-b0a2-42d4-9e31-7db405cc5544 service swift] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:54:30.039562 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1099/2216] 10.222.0.71 () {60 vars in 1384 bytes} [Fri Aug 26 08:54:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 08:54:36.378015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c489ba-88da-4f97-89a2-706fd0f0edc1 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:54:36.378694 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c489ba-88da-4f97-89a2-706fd0f0edc1 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:54:36.379038 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c489ba-88da-4f97-89a2-706fd0f0edc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:54:36.425230 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-11c489ba-88da-4f97-89a2-706fd0f0edc1 None None] MFA Rules not processed for user `df07cffd82c74621855c8b8d7bfa989d`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:54:36.427694 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-11c489ba-88da-4f97-89a2-706fd0f0edc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:54:36.466263 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1118/2217] 10.222.0.71 () {58 vars in 1009 bytes} [Fri Aug 26 08:54:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:54:36.480059 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60eb20d-b0a2-42d4-9e31-7db405cc5544 service swift] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:54:36.508594 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60eb20d-b0a2-42d4-9e31-7db405cc5544 service swift] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:54:36.524798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbc9e0-2afc-4324-a686-b5eb1570966c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1723e961375b45b7b3bd99ddeb69683f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:54:36.525803 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbc9e0-2afc-4324-a686-b5eb1570966c service swift] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:54:36.526239 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbc9e0-2afc-4324-a686-b5eb1570966c service swift] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:54:36.526667 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbc9e0-2afc-4324-a686-b5eb1570966c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:54:36.560092 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cbc9e0-2afc-4324-a686-b5eb1570966c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:54:36.562871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cbc9e0-2afc-4324-a686-b5eb1570966c service swift] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:54:36.592359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1100/2218] 10.222.0.71 () {60 vars in 1384 bytes} [Fri Aug 26 08:54:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 08:54:37.640881 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c489ba-88da-4f97-89a2-706fd0f0edc1 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:54:37.645125 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-11c489ba-88da-4f97-89a2-706fd0f0edc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:54:37.877879 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c489ba-88da-4f97-89a2-706fd0f0edc1 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:54:37.892386 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14611c1-75a2-47f8-afed-670239951c71 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf5e2b4f5e440a7942adea3d51cec75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:54:37.893615 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14611c1-75a2-47f8-afed-670239951c71 service placement] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:54:37.894111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14611c1-75a2-47f8-afed-670239951c71 service placement] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:54:37.894759 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14611c1-75a2-47f8-afed-670239951c71 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:54:37.898745 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e14611c1-75a2-47f8-afed-670239951c71 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:54:37.934063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14611c1-75a2-47f8-afed-670239951c71 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:54:37.941145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14611c1-75a2-47f8-afed-670239951c71 service placement] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:54:37.976886 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1119/2219] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:54:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:54:46.123720 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbc9e0-2afc-4324-a686-b5eb1570966c service swift] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:54:46.126082 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b7cbc9e0-2afc-4324-a686-b5eb1570966c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:54:46.360185 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbc9e0-2afc-4324-a686-b5eb1570966c service swift] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:54:46.381615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868468d7-29b3-4367-aa13-6257280720da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:54:46.382846 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868468d7-29b3-4367-aa13-6257280720da admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:54:46.383717 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868468d7-29b3-4367-aa13-6257280720da admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:54:46.384318 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868468d7-29b3-4367-aa13-6257280720da admin admin] PATH_INFO: `/v3/users/100dde650f224b6f9654d7c8bb36d367` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:54:46.388169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868468d7-29b3-4367-aa13-6257280720da admin admin] RBAC: Authorizing `identity:delete_user(user_id=100dde650f224b6f9654d7c8bb36d367)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:54:46.391216 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868468d7-29b3-4367-aa13-6257280720da admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:54:46.588964 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-868468d7-29b3-4367-aa13-6257280720da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '100dde650f224b6f9654d7c8bb36d367'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:54:46.613196 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-868468d7-29b3-4367-aa13-6257280720da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '100dde650f224b6f9654d7c8bb36d367'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:54:46.660318 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-868468d7-29b3-4367-aa13-6257280720da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '100dde650f224b6f9654d7c8bb36d367'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:54:46.667438 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-868468d7-29b3-4367-aa13-6257280720da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '100dde650f224b6f9654d7c8bb36d367'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:54:46.668493 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-868468d7-29b3-4367-aa13-6257280720da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '100dde650f224b6f9654d7c8bb36d367'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:54:46.717586 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:54:46.742202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1101/2220] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:54:46 2022] DELETE /identity/v3/users/100dde650f224b6f9654d7c8bb36d367 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:54:46.747399 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78821b1f-c006-47b9-892c-84cead456d68 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:54:46.748078 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78821b1f-c006-47b9-892c-84cead456d68 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:54:46.748437 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78821b1f-c006-47b9-892c-84cead456d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:54:46.808970 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-78821b1f-c006-47b9-892c-84cead456d68 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:54:46.811198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-78821b1f-c006-47b9-892c-84cead456d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:54:47.032458 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1120/2221] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:54:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:54:47.059243 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868468d7-29b3-4367-aa13-6257280720da admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:54:47.062224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-868468d7-29b3-4367-aa13-6257280720da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:54:47.303226 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868468d7-29b3-4367-aa13-6257280720da admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:54:47.315566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4865cb-22b7-4e12-a02e-4762d6052acd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:54:47.316257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4865cb-22b7-4e12-a02e-4762d6052acd service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:54:47.319789 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4865cb-22b7-4e12-a02e-4762d6052acd service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:54:47.319789 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4865cb-22b7-4e12-a02e-4762d6052acd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:54:47.339012 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4865cb-22b7-4e12-a02e-4762d6052acd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:54:47.343605 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4865cb-22b7-4e12-a02e-4762d6052acd service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:54:47.395012 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1102/2222] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:54:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:54:48.460785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78821b1f-c006-47b9-892c-84cead456d68 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:54:48.468076 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-78821b1f-c006-47b9-892c-84cead456d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:54:48.504970 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78821b1f-c006-47b9-892c-84cead456d68 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:54:48.513529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8e0ec-3a10-41ef-801c-2b75bf808dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:54:48.514962 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8e0ec-3a10-41ef-801c-2b75bf808dce admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:54:48.515619 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8e0ec-3a10-41ef-801c-2b75bf808dce admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:54:48.516293 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8e0ec-3a10-41ef-801c-2b75bf808dce admin admin] PATH_INFO: `/v3/projects/30fad5203b0147acb083e51ed8eed9bf` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:54:48.521022 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8e0ec-3a10-41ef-801c-2b75bf808dce admin admin] RBAC: Authorizing `identity:delete_project(project_id=30fad5203b0147acb083e51ed8eed9bf)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:54:48.524872 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8e0ec-3a10-41ef-801c-2b75bf808dce admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:54:48.613680 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:54:48.691850 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b2d8e0ec-3a10-41ef-801c-2b75bf808dce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30fad5203b0147acb083e51ed8eed9bf', 'request_id': 'req-b2d8e0ec-3a10-41ef-801c-2b75bf808dce'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:54:48.706327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b2d8e0ec-3a10-41ef-801c-2b75bf808dce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30fad5203b0147acb083e51ed8eed9bf', 'request_id': 'req-b2d8e0ec-3a10-41ef-801c-2b75bf808dce'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:54:48.729748 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b2d8e0ec-3a10-41ef-801c-2b75bf808dce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30fad5203b0147acb083e51ed8eed9bf', 'request_id': 'req-b2d8e0ec-3a10-41ef-801c-2b75bf808dce'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:54:48.781408 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b2d8e0ec-3a10-41ef-801c-2b75bf808dce admin admin] The token cache is being invalidate because project 30fad5203b0147acb083e51ed8eed9bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:54:48.782295 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b2d8e0ec-3a10-41ef-801c-2b75bf808dce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:54:48.783671 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b2d8e0ec-3a10-41ef-801c-2b75bf808dce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:54:48.796572 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cedaf-7f03-4feb-aeac-3c773d765419 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:54:48.798629 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cedaf-7f03-4feb-aeac-3c773d765419 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:54:48.799436 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cedaf-7f03-4feb-aeac-3c773d765419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:54:48.804889 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1121/2223] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:54:48 2022] DELETE /identity/v3/projects/30fad5203b0147acb083e51ed8eed9bf => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:54:48.845251 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-047cedaf-7f03-4feb-aeac-3c773d765419 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:54:48.848441 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-047cedaf-7f03-4feb-aeac-3c773d765419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:54:49.136736 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b18b6b-a519-4625-8617-5bc7d7321952 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:54:49.137608 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b18b6b-a519-4625-8617-5bc7d7321952 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:54:49.138355 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b18b6b-a519-4625-8617-5bc7d7321952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:54:49.147750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1103/2224] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:54:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:54:49.188762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-21b18b6b-a519-4625-8617-5bc7d7321952 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:54:49.192487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-21b18b6b-a519-4625-8617-5bc7d7321952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:54:49.225138 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1122/2225] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:54:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:54:49.226200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047cedaf-7f03-4feb-aeac-3c773d765419 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:54:49.249688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047cedaf-7f03-4feb-aeac-3c773d765419 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:54:49.264547 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c5f4a-d7d1-4421-98b2-c8d8a55f83ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:54:49.267185 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c5f4a-d7d1-4421-98b2-c8d8a55f83ce admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:54:49.268018 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c5f4a-d7d1-4421-98b2-c8d8a55f83ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:54:49.268840 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c5f4a-d7d1-4421-98b2-c8d8a55f83ce admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:54:49.270407 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c5f4a-d7d1-4421-98b2-c8d8a55f83ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:54:49.271196 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c5f4a-d7d1-4421-98b2-c8d8a55f83ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:54:49.276443 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c5f4a-d7d1-4421-98b2-c8d8a55f83ce admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:54:49.348655 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1104/2226] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:54:49.351249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b18b6b-a519-4625-8617-5bc7d7321952 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:54:49.423249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b18b6b-a519-4625-8617-5bc7d7321952 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:54:49.449227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82f320-3a4e-4cd5-8b8c-681a6479233b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:54:49.450244 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82f320-3a4e-4cd5-8b8c-681a6479233b admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:54:49.453506 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82f320-3a4e-4cd5-8b8c-681a6479233b admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:54:49.453970 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82f320-3a4e-4cd5-8b8c-681a6479233b admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:54:49.455205 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d82f320-3a4e-4cd5-8b8c-681a6479233b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:54:49.466472 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d82f320-3a4e-4cd5-8b8c-681a6479233b admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:54:49.615044 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1123/2227] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:54:49 2022] POST /identity/v3/projects => generated 373 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:54:49.621803 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c5f4a-d7d1-4421-98b2-c8d8a55f83ce admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:54:49.668945 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c5f4a-d7d1-4421-98b2-c8d8a55f83ce admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:54:50.220315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d74de-b0a6-4758-986e-9c0ad90e4bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:54:50.228632 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d74de-b0a6-4758-986e-9c0ad90e4bfb admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:54:50.229077 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d74de-b0a6-4758-986e-9c0ad90e4bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:54:50.231603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d74de-b0a6-4758-986e-9c0ad90e4bfb admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:54:50.242728 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d74de-b0a6-4758-986e-9c0ad90e4bfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:54:50.252153 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d74de-b0a6-4758-986e-9c0ad90e4bfb admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:54:50.359896 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1105/2228] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:54:49 2022] POST /identity/v3/users => generated 287 bytes in 739 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:54:50.368043 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82f320-3a4e-4cd5-8b8c-681a6479233b admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:54:50.433957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82f320-3a4e-4cd5-8b8c-681a6479233b admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:54:50.481323 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fae6d-bb94-4bcd-b712-571796fe0556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:54:50.482370 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fae6d-bb94-4bcd-b712-571796fe0556 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:54:50.482843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fae6d-bb94-4bcd-b712-571796fe0556 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:54:50.483397 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fae6d-bb94-4bcd-b712-571796fe0556 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:54:50.484618 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4fae6d-bb94-4bcd-b712-571796fe0556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:54:50.501561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4fae6d-bb94-4bcd-b712-571796fe0556 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:54:50.559126 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1124/2229] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:54:50.564777 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d74de-b0a6-4758-986e-9c0ad90e4bfb admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:54:50.585724 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d74de-b0a6-4758-986e-9c0ad90e4bfb admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:54:50.611120 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44886a08-cc18-4fa8-b0f8-30e23466a9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:54:50.611561 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44886a08-cc18-4fa8-b0f8-30e23466a9d1 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:54:50.611781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44886a08-cc18-4fa8-b0f8-30e23466a9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:54:50.612046 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44886a08-cc18-4fa8-b0f8-30e23466a9d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:54:50.612746 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44886a08-cc18-4fa8-b0f8-30e23466a9d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:54:50.614752 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44886a08-cc18-4fa8-b0f8-30e23466a9d1 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:54:50.657279 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1106/2230] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:54:50.663425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fae6d-bb94-4bcd-b712-571796fe0556 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:54:50.679540 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fae6d-bb94-4bcd-b712-571796fe0556 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:54:50.716590 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e28b2-1824-4feb-9420-a9ee6056dc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:54:50.718744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e28b2-1824-4feb-9420-a9ee6056dc31 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:54:50.719311 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e28b2-1824-4feb-9420-a9ee6056dc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:54:50.719792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e28b2-1824-4feb-9420-a9ee6056dc31 admin admin] PATH_INFO: `/v3/projects/d9c2937d8e1243c185dab8e97f0a6e90/users/3d3ea95f39e24567af6f1be8f7e2a60b/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:54:50.789904 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e28b2-1824-4feb-9420-a9ee6056dc31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9c2937d8e1243c185dab8e97f0a6e90, user_id=3d3ea95f39e24567af6f1be8f7e2a60b, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:54:50.800659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e28b2-1824-4feb-9420-a9ee6056dc31 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:54:50.858932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1125/2231] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:54:50 2022] PUT /identity/v3/projects/d9c2937d8e1243c185dab8e97f0a6e90/users/3d3ea95f39e24567af6f1be8f7e2a60b/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:54:50.868875 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fafd35-2a31-4cea-b5d6-7add6b882319 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:54:50.869319 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fafd35-2a31-4cea-b5d6-7add6b882319 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:54:50.869744 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fafd35-2a31-4cea-b5d6-7add6b882319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:54:50.893579 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:54:50.969643 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-a1fafd35-2a31-4cea-b5d6-7add6b882319 None None] MFA Rules not processed for user `3d3ea95f39e24567af6f1be8f7e2a60b`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:54:50.972261 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a1fafd35-2a31-4cea-b5d6-7add6b882319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:54:51.214057 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1107/2232] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:54:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:54:51.241513 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e28b2-1824-4feb-9420-a9ee6056dc31 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:54:51.244885 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8d7e28b2-1824-4feb-9420-a9ee6056dc31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:54:51.441675 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e28b2-1824-4feb-9420-a9ee6056dc31 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:54:51.458155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410cc8fc-5875-4518-8cd0-d59ed191bd32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:54:51.458874 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410cc8fc-5875-4518-8cd0-d59ed191bd32 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:54:51.459179 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410cc8fc-5875-4518-8cd0-d59ed191bd32 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:54:51.459455 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410cc8fc-5875-4518-8cd0-d59ed191bd32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:54:51.504891 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410cc8fc-5875-4518-8cd0-d59ed191bd32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:54:51.506364 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410cc8fc-5875-4518-8cd0-d59ed191bd32 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:54:51.518454 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1126/2233] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:54:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:54:52.559303 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416523f-4a62-4358-bee2-5a2e841f0b96 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:54:52.560194 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416523f-4a62-4358-bee2-5a2e841f0b96 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:54:52.560194 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416523f-4a62-4358-bee2-5a2e841f0b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:54:52.672338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-1416523f-4a62-4358-bee2-5a2e841f0b96 None None] MFA Rules not processed for user `3aced0ed17b946f9961d87a1d4446c37`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:54:52.674138 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1416523f-4a62-4358-bee2-5a2e841f0b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:54:52.801151 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1108/2234] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:54:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:54:53.738304 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0d5c1-4693-42d4-983b-a044ebf961c0 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:54:53.739192 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0d5c1-4693-42d4-983b-a044ebf961c0 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:54:53.739300 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0d5c1-4693-42d4-983b-a044ebf961c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:54:53.820686 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-67f0d5c1-4693-42d4-983b-a044ebf961c0 None None] MFA Rules not processed for user `3d3ea95f39e24567af6f1be8f7e2a60b`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:54:53.824378 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-67f0d5c1-4693-42d4-983b-a044ebf961c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:54:53.861950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1127/2235] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:54:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:55:01.638185 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8714f2-d448-4a61-b912-1e9c7481ba1d None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:01.639168 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8714f2-d448-4a61-b912-1e9c7481ba1d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:01.639168 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8714f2-d448-4a61-b912-1e9c7481ba1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:01.675718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-1b8714f2-d448-4a61-b912-1e9c7481ba1d None None] MFA Rules not processed for user `df07cffd82c74621855c8b8d7bfa989d`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:55:01.676941 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1b8714f2-d448-4a61-b912-1e9c7481ba1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:55:01.747474 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1109/2236] 10.222.0.71 () {58 vars in 967 bytes} [Fri Aug 26 08:55:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:55:01.761824 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0d5c1-4693-42d4-983b-a044ebf961c0 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:01.763566 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-67f0d5c1-4693-42d4-983b-a044ebf961c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:55:01.839957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:55:01.869431 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0d5c1-4693-42d4-983b-a044ebf961c0 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:01.875656 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942db835-2429-4902-b86d-de4cadb820c9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1723e961375b45b7b3bd99ddeb69683f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:01.876443 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942db835-2429-4902-b86d-de4cadb820c9 service swift] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:01.876957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942db835-2429-4902-b86d-de4cadb820c9 service swift] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:01.877846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942db835-2429-4902-b86d-de4cadb820c9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:01.892260 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942db835-2429-4902-b86d-de4cadb820c9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:01.895232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942db835-2429-4902-b86d-de4cadb820c9 service swift] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:01.918767 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1128/2237] 10.222.0.71 () {60 vars in 1384 bytes} [Fri Aug 26 08:55:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 08:55:05.134535 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8714f2-d448-4a61-b912-1e9c7481ba1d None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:05.137052 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1b8714f2-d448-4a61-b912-1e9c7481ba1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:55:05.271043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:55:05.300474 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8714f2-d448-4a61-b912-1e9c7481ba1d None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:05.306115 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2037234-be40-4005-ad25-523ca073322f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:05.306785 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2037234-be40-4005-ad25-523ca073322f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:05.307115 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2037234-be40-4005-ad25-523ca073322f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:05.307451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2037234-be40-4005-ad25-523ca073322f admin admin] PATH_INFO: `/v3/users/230efbe75687494fa44777325c8acb4e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:05.308865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2037234-be40-4005-ad25-523ca073322f admin admin] RBAC: Authorizing `identity:delete_user(user_id=230efbe75687494fa44777325c8acb4e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:05.311832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2037234-be40-4005-ad25-523ca073322f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:05.412182 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b2037234-be40-4005-ad25-523ca073322f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '230efbe75687494fa44777325c8acb4e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:55:05.427426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b2037234-be40-4005-ad25-523ca073322f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '230efbe75687494fa44777325c8acb4e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:55:05.440377 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b2037234-be40-4005-ad25-523ca073322f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '230efbe75687494fa44777325c8acb4e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:55:05.452220 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b2037234-be40-4005-ad25-523ca073322f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '230efbe75687494fa44777325c8acb4e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:55:05.455448 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b2037234-be40-4005-ad25-523ca073322f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '230efbe75687494fa44777325c8acb4e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:55:05.498536 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1110/2238] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:55:05 2022] DELETE /identity/v3/users/230efbe75687494fa44777325c8acb4e => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:55:05.504186 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e459004-2a55-4697-8735-dd986990b0eb None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:05.504794 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e459004-2a55-4697-8735-dd986990b0eb None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:05.504794 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e459004-2a55-4697-8735-dd986990b0eb None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:05.506234 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1129/2239] 10.222.0.71 () {54 vars in 921 bytes} [Fri Aug 26 08:55:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:55:05.521740 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6240280-780f-43c4-a382-37334efa046e None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:05.522040 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6240280-780f-43c4-a382-37334efa046e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:05.522323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6240280-780f-43c4-a382-37334efa046e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:05.549632 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-e6240280-780f-43c4-a382-37334efa046e None None] MFA Rules not processed for user `f16d1768715a41648af3fe6afadd7477`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:55:05.551915 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e6240280-780f-43c4-a382-37334efa046e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:55:05.698973 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1111/2240] 10.222.0.71 () {58 vars in 1018 bytes} [Fri Aug 26 08:55:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:55:05.717338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e459004-2a55-4697-8735-dd986990b0eb None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:05.720224 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4e459004-2a55-4697-8735-dd986990b0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:55:05.918774 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6240280-780f-43c4-a382-37334efa046e None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:05.922432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e6240280-780f-43c4-a382-37334efa046e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:55:05.928278 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e459004-2a55-4697-8735-dd986990b0eb None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:05.942637 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feee3ab-dd76-4e2b-80ac-b9a697d01b91 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d178c717a3e49319545e70cb58cbbed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:05.943603 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feee3ab-dd76-4e2b-80ac-b9a697d01b91 service glance] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:05.944035 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feee3ab-dd76-4e2b-80ac-b9a697d01b91 service glance] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:05.947505 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feee3ab-dd76-4e2b-80ac-b9a697d01b91 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:05.967915 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feee3ab-dd76-4e2b-80ac-b9a697d01b91 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:05.973257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feee3ab-dd76-4e2b-80ac-b9a697d01b91 service glance] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:06.003172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1130/2241] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:55:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:55:06.043944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:55:06.055450 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6240280-780f-43c4-a382-37334efa046e None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:06.068295 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6c5e6-deff-4bc2-bbd1-971a2482ff67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:06.070112 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6c5e6-deff-4bc2-bbd1-971a2482ff67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:06.070518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6c5e6-deff-4bc2-bbd1-971a2482ff67 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:06.070921 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6c5e6-deff-4bc2-bbd1-971a2482ff67 admin admin] PATH_INFO: `/v3/projects/196b8882c5344c8380fecba536f39214` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:06.072290 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6c5e6-deff-4bc2-bbd1-971a2482ff67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=196b8882c5344c8380fecba536f39214)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:06.074736 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6c5e6-deff-4bc2-bbd1-971a2482ff67 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:06.122626 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3558e-4ec7-4e7f-a055-108a0133b4a2 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:06.123591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3558e-4ec7-4e7f-a055-108a0133b4a2 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:06.124249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3558e-4ec7-4e7f-a055-108a0133b4a2 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:06.126475 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1131/2242] 10.222.0.71 () {54 vars in 921 bytes} [Fri Aug 26 08:55:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 08:55:06.135280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c2a3e-4557-4c6a-b3eb-1e88661c19a5 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:06.136134 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c2a3e-4557-4c6a-b3eb-1e88661c19a5 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:06.136604 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c2a3e-4557-4c6a-b3eb-1e88661c19a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:06.159575 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c3f6c5e6-deff-4bc2-bbd1-971a2482ff67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '196b8882c5344c8380fecba536f39214', 'request_id': 'req-c3f6c5e6-deff-4bc2-bbd1-971a2482ff67'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:55:06.176688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c3f6c5e6-deff-4bc2-bbd1-971a2482ff67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '196b8882c5344c8380fecba536f39214', 'request_id': 'req-c3f6c5e6-deff-4bc2-bbd1-971a2482ff67'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:55:06.180014 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ba2c2a3e-4557-4c6a-b3eb-1e88661c19a5 None None] MFA Rules not processed for user `4aa8938a504c4c4f8815478e520482d8`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:55:06.183494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ba2c2a3e-4557-4c6a-b3eb-1e88661c19a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:55:06.186516 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c3f6c5e6-deff-4bc2-bbd1-971a2482ff67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '196b8882c5344c8380fecba536f39214', 'request_id': 'req-c3f6c5e6-deff-4bc2-bbd1-971a2482ff67'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:55:06.208487 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c3f6c5e6-deff-4bc2-bbd1-971a2482ff67 admin admin] The token cache is being invalidate because project 196b8882c5344c8380fecba536f39214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:55:06.210943 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c3f6c5e6-deff-4bc2-bbd1-971a2482ff67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:55:06.214057 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c3f6c5e6-deff-4bc2-bbd1-971a2482ff67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:55:06.219301 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1112/2243] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:55:05 2022] DELETE /identity/v3/projects/196b8882c5344c8380fecba536f39214 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:55:06.226106 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42061b8e-a3b6-45df-9a8f-be5dacb1757d None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:06.227389 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42061b8e-a3b6-45df-9a8f-be5dacb1757d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:06.229116 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42061b8e-a3b6-45df-9a8f-be5dacb1757d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:06.261279 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-42061b8e-a3b6-45df-9a8f-be5dacb1757d None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:55:06.262747 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-42061b8e-a3b6-45df-9a8f-be5dacb1757d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:55:06.292238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1132/2244] 10.222.0.71 () {58 vars in 1018 bytes} [Fri Aug 26 08:55:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:55:06.325621 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c2a3e-4557-4c6a-b3eb-1e88661c19a5 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:06.329516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ba2c2a3e-4557-4c6a-b3eb-1e88661c19a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:55:06.354238 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1113/2245] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:55:06.364605 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb253a3-7d4c-4eb0-bd2d-8be5a2692c0f None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:06.365015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb253a3-7d4c-4eb0-bd2d-8be5a2692c0f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:06.365375 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb253a3-7d4c-4eb0-bd2d-8be5a2692c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:06.367269 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c2a3e-4557-4c6a-b3eb-1e88661c19a5 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:06.381799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6024f-3508-42e1-bad8-b55542f9ab7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:06.382872 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6024f-3508-42e1-bad8-b55542f9ab7a service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:06.383461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6024f-3508-42e1-bad8-b55542f9ab7a service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:06.386346 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6024f-3508-42e1-bad8-b55542f9ab7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:06.401574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d6024f-3508-42e1-bad8-b55542f9ab7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:06.405415 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d6024f-3508-42e1-bad8-b55542f9ab7a service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:06.421199 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-1bb253a3-7d4c-4eb0-bd2d-8be5a2692c0f None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:55:06.421698 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1bb253a3-7d4c-4eb0-bd2d-8be5a2692c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:55:06.424658 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1133/2246] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:55:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:55:06.447069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1114/2247] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:55:06.449587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6024f-3508-42e1-bad8-b55542f9ab7a service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:06.471269 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6024f-3508-42e1-bad8-b55542f9ab7a service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:06.504684 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37d7fbb-c2f7-45bb-9705-0e6302cd7510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:06.506589 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d7fbb-c2f7-45bb-9705-0e6302cd7510 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:06.507270 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d7fbb-c2f7-45bb-9705-0e6302cd7510 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:06.507942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37d7fbb-c2f7-45bb-9705-0e6302cd7510 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:06.509412 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d7fbb-c2f7-45bb-9705-0e6302cd7510 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:55:06.510313 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d7fbb-c2f7-45bb-9705-0e6302cd7510 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:06.513411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37d7fbb-c2f7-45bb-9705-0e6302cd7510 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:06.553627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1134/2248] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:55:06.564105 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb253a3-7d4c-4eb0-bd2d-8be5a2692c0f None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:06.578126 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb253a3-7d4c-4eb0-bd2d-8be5a2692c0f None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:06.594518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc6089-db22-42eb-8153-6c4d6762f30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:06.595209 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc6089-db22-42eb-8153-6c4d6762f30b admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:06.595682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc6089-db22-42eb-8153-6c4d6762f30b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:06.596017 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc6089-db22-42eb-8153-6c4d6762f30b admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:06.596783 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc6089-db22-42eb-8153-6c4d6762f30b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:06.598789 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc6089-db22-42eb-8153-6c4d6762f30b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:06.634244 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e94ea0-612b-4619-8b72-ccdeac73c013 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:06.635423 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e94ea0-612b-4619-8b72-ccdeac73c013 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:06.635423 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e94ea0-612b-4619-8b72-ccdeac73c013 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:06.638584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1135/2249] 10.222.0.71 () {54 vars in 921 bytes} [Fri Aug 26 08:55:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:55:06.655065 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89cc5b-debf-4c4c-8049-fc743164fe67 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:06.655615 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89cc5b-debf-4c4c-8049-fc743164fe67 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:06.655943 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd89cc5b-debf-4c4c-8049-fc743164fe67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:06.675475 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1115/2250] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:55:06 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:55:06.685907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc6089-db22-42eb-8153-6c4d6762f30b admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:06.713087 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-dd89cc5b-debf-4c4c-8049-fc743164fe67 None None] MFA Rules not processed for user `f16d1768715a41648af3fe6afadd7477`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:55:06.714822 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-dd89cc5b-debf-4c4c-8049-fc743164fe67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:55:06.848367 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:55:06.857366 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc6089-db22-42eb-8153-6c4d6762f30b admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:06.884997 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1a8ca-ed42-4ada-8f0c-c2dc21a2d7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:06.885486 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1a8ca-ed42-4ada-8f0c-c2dc21a2d7af admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:06.885870 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1a8ca-ed42-4ada-8f0c-c2dc21a2d7af admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:06.887008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1a8ca-ed42-4ada-8f0c-c2dc21a2d7af admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:06.887656 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1a8ca-ed42-4ada-8f0c-c2dc21a2d7af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:06.890149 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1a8ca-ed42-4ada-8f0c-c2dc21a2d7af admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:06.890404 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:55:06.940218 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1136/2251] 10.222.0.71 () {58 vars in 1018 bytes} [Fri Aug 26 08:55:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:55:06.957701 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1116/2252] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:55:06 2022] POST /identity/v3/users => generated 293 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:55:06.963924 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89cc5b-debf-4c4c-8049-fc743164fe67 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:06.983798 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd89cc5b-debf-4c4c-8049-fc743164fe67 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:06.989358 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2ee94-1f2f-4547-981c-b2b72e2d15bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:06.990071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2ee94-1f2f-4547-981c-b2b72e2d15bc admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:06.990362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2ee94-1f2f-4547-981c-b2b72e2d15bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:06.990643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2ee94-1f2f-4547-981c-b2b72e2d15bc admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:06.991543 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e2ee94-1f2f-4547-981c-b2b72e2d15bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:06.993505 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e2ee94-1f2f-4547-981c-b2b72e2d15bc admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:07.010289 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1137/2253] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:55:07.015597 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1a8ca-ed42-4ada-8f0c-c2dc21a2d7af admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:07.029366 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2ee94-1f2f-4547-981c-b2b72e2d15bc admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:07.044670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1a8ca-ed42-4ada-8f0c-c2dc21a2d7af admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:07.067718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbac2b5-c4bb-45f2-aef6-18000bb1a1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:07.069415 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbac2b5-c4bb-45f2-aef6-18000bb1a1b4 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:07.070407 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbac2b5-c4bb-45f2-aef6-18000bb1a1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:07.073314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbac2b5-c4bb-45f2-aef6-18000bb1a1b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:07.074987 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbac2b5-c4bb-45f2-aef6-18000bb1a1b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:07.079314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbac2b5-c4bb-45f2-aef6-18000bb1a1b4 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:07.107126 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1117/2254] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:55:07.112724 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbac2b5-c4bb-45f2-aef6-18000bb1a1b4 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:07.133923 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbac2b5-c4bb-45f2-aef6-18000bb1a1b4 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:07.146223 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c147b9-0f8d-4041-9679-a887e4386a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:07.147421 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c147b9-0f8d-4041-9679-a887e4386a02 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:07.147900 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c147b9-0f8d-4041-9679-a887e4386a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:07.149051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c147b9-0f8d-4041-9679-a887e4386a02 admin admin] PATH_INFO: `/v3/projects/3d18cb354a0f46faba6c00b89b4e4b1c/users/2cee3394dbdd4cb3a52005ee357d3be7/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:07.150195 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2ee94-1f2f-4547-981c-b2b72e2d15bc admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:07.160199 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2796bb9-4656-4de3-9f9f-15cd7ec5a0a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:07.160953 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2796bb9-4656-4de3-9f9f-15cd7ec5a0a7 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:07.161238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2796bb9-4656-4de3-9f9f-15cd7ec5a0a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:07.161514 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2796bb9-4656-4de3-9f9f-15cd7ec5a0a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:07.178083 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2796bb9-4656-4de3-9f9f-15cd7ec5a0a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:07.182549 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2796bb9-4656-4de3-9f9f-15cd7ec5a0a7 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:07.196036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c147b9-0f8d-4041-9679-a887e4386a02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d18cb354a0f46faba6c00b89b4e4b1c, user_id=2cee3394dbdd4cb3a52005ee357d3be7, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:07.201799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c147b9-0f8d-4041-9679-a887e4386a02 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:07.209798 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1138/2255] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:55:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:55:07.251889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1118/2256] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:55:07 2022] PUT /identity/v3/projects/3d18cb354a0f46faba6c00b89b4e4b1c/users/2cee3394dbdd4cb3a52005ee357d3be7/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:55:07.253650 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c062412-8e49-4392-bb43-92850caa5aa1 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:07.254207 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c062412-8e49-4392-bb43-92850caa5aa1 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:07.254907 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c062412-8e49-4392-bb43-92850caa5aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:07.283856 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-0c062412-8e49-4392-bb43-92850caa5aa1 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:55:07.285495 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-0c062412-8e49-4392-bb43-92850caa5aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:55:07.459454 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1139/2257] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:55:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:55:07.480116 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c147b9-0f8d-4041-9679-a887e4386a02 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:07.584807 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c147b9-0f8d-4041-9679-a887e4386a02 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:07.591691 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea7420-4fd3-4517-88d0-dd0fe061b7dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:07.592410 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea7420-4fd3-4517-88d0-dd0fe061b7dc service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:07.592812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea7420-4fd3-4517-88d0-dd0fe061b7dc service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:07.593099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea7420-4fd3-4517-88d0-dd0fe061b7dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:07.610364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea7420-4fd3-4517-88d0-dd0fe061b7dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:07.614658 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea7420-4fd3-4517-88d0-dd0fe061b7dc service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:07.643478 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1119/2258] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:55:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:55:17.445168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=62 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:55:17.622577 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=44,keystone:INSERT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:55:17.701433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e653d-afae-41f5-ace8-87f0819bd803 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:17.701433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e653d-afae-41f5-ace8-87f0819bd803 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:17.701966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e653d-afae-41f5-ace8-87f0819bd803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:17.762310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-d05e653d-afae-41f5-ace8-87f0819bd803 None None] MFA Rules not processed for user `2cee3394dbdd4cb3a52005ee357d3be7`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:55:17.769730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d05e653d-afae-41f5-ace8-87f0819bd803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:55:17.889628 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1140/2259] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:55:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:55:17.899953 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea7420-4fd3-4517-88d0-dd0fe061b7dc service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:17.947923 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea7420-4fd3-4517-88d0-dd0fe061b7dc service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:17.960151 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffed3fa-0223-4241-a5eb-eb17a95739b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:17.962879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffed3fa-0223-4241-a5eb-eb17a95739b6 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:17.963561 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffed3fa-0223-4241-a5eb-eb17a95739b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:17.964206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffed3fa-0223-4241-a5eb-eb17a95739b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:17.965602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffed3fa-0223-4241-a5eb-eb17a95739b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:17.970988 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffed3fa-0223-4241-a5eb-eb17a95739b6 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:18.068317 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1120/2260] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:55:17 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:55:18.073040 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e653d-afae-41f5-ace8-87f0819bd803 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:18.097594 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e653d-afae-41f5-ace8-87f0819bd803 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:18.313080 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a3e27-7c47-4856-b224-55c41290808c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:18.314442 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a3e27-7c47-4856-b224-55c41290808c admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:18.315134 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a3e27-7c47-4856-b224-55c41290808c admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:18.315555 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a3e27-7c47-4856-b224-55c41290808c admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:18.319804 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a3e27-7c47-4856-b224-55c41290808c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:18.331989 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a3e27-7c47-4856-b224-55c41290808c admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:18.427239 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffed3fa-0223-4241-a5eb-eb17a95739b6 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:18.438026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1141/2261] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:55:18 2022] POST /identity/v3/users => generated 299 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:55:18.467409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381e459-75fc-498d-af99-28feb825a470 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:18.467409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381e459-75fc-498d-af99-28feb825a470 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:18.467409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381e459-75fc-498d-af99-28feb825a470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:18.473272 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffed3fa-0223-4241-a5eb-eb17a95739b6 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:18.495668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff70f1-4955-4434-9400-53501d83af0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:18.497961 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff70f1-4955-4434-9400-53501d83af0e admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:18.498329 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff70f1-4955-4434-9400-53501d83af0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:18.498842 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff70f1-4955-4434-9400-53501d83af0e admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:18.507565 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff70f1-4955-4434-9400-53501d83af0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:18.514803 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff70f1-4955-4434-9400-53501d83af0e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:18.553547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-3381e459-75fc-498d-af99-28feb825a470 None None] MFA Rules not processed for user `df07cffd82c74621855c8b8d7bfa989d`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:55:18.561194 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3381e459-75fc-498d-af99-28feb825a470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:55:18.591987 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1121/2262] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:55:18.595028 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff70f1-4955-4434-9400-53501d83af0e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:18.679883 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff70f1-4955-4434-9400-53501d83af0e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:18.701098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc15e92-6dd9-4a73-9f4b-ee8be7ba240e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:18.702278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc15e92-6dd9-4a73-9f4b-ee8be7ba240e admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:18.702778 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc15e92-6dd9-4a73-9f4b-ee8be7ba240e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:18.703322 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc15e92-6dd9-4a73-9f4b-ee8be7ba240e admin admin] PATH_INFO: `/v3/projects/5ffc549da2e2446889e2f0c0cc66e177/users/ef15b1a9f42a402d92f65fd4971f2e81/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:18.706210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1142/2263] 10.222.0.71 () {58 vars in 1009 bytes} [Fri Aug 26 08:55:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:55:18.729122 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381e459-75fc-498d-af99-28feb825a470 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:18.735604 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3381e459-75fc-498d-af99-28feb825a470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:55:18.781157 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc15e92-6dd9-4a73-9f4b-ee8be7ba240e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ffc549da2e2446889e2f0c0cc66e177, user_id=ef15b1a9f42a402d92f65fd4971f2e81, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:18.792896 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc15e92-6dd9-4a73-9f4b-ee8be7ba240e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:18.798299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:55:18.843706 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1122/2264] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:55:18 2022] PUT /identity/v3/projects/5ffc549da2e2446889e2f0c0cc66e177/users/ef15b1a9f42a402d92f65fd4971f2e81/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:55:18.849652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc15e92-6dd9-4a73-9f4b-ee8be7ba240e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:18.941404 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381e459-75fc-498d-af99-28feb825a470 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:19.090224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc15e92-6dd9-4a73-9f4b-ee8be7ba240e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:19.110986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff958af-cf14-4caa-a099-a8fb7fee0f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:19.115813 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff958af-cf14-4caa-a099-a8fb7fee0f6e admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:19.116342 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff958af-cf14-4caa-a099-a8fb7fee0f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:19.116894 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff958af-cf14-4caa-a099-a8fb7fee0f6e admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:19.120585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff958af-cf14-4caa-a099-a8fb7fee0f6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:19.132781 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff958af-cf14-4caa-a099-a8fb7fee0f6e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:19.181790 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1123/2265] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 08:55:19.186389 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff958af-cf14-4caa-a099-a8fb7fee0f6e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:19.218073 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a218d-5383-465b-b10b-75b0cddcdcea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1723e961375b45b7b3bd99ddeb69683f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:19.219144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a218d-5383-465b-b10b-75b0cddcdcea service swift] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:19.220404 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a218d-5383-465b-b10b-75b0cddcdcea service swift] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:19.220877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a218d-5383-465b-b10b-75b0cddcdcea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:19.225703 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff958af-cf14-4caa-a099-a8fb7fee0f6e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:19.247485 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45afc521-1896-4eb0-92d6-bde8982c8bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:19.248955 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afc521-1896-4eb0-92d6-bde8982c8bec admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:19.249717 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afc521-1896-4eb0-92d6-bde8982c8bec admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:19.250327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45afc521-1896-4eb0-92d6-bde8982c8bec admin admin] PATH_INFO: `/v3/projects/5ffc549da2e2446889e2f0c0cc66e177/users/ef15b1a9f42a402d92f65fd4971f2e81/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:19.255968 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45afc521-1896-4eb0-92d6-bde8982c8bec admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ffc549da2e2446889e2f0c0cc66e177, user_id=ef15b1a9f42a402d92f65fd4971f2e81, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:19.269911 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45afc521-1896-4eb0-92d6-bde8982c8bec admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:19.323563 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1124/2266] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:55:19 2022] PUT /identity/v3/projects/5ffc549da2e2446889e2f0c0cc66e177/users/ef15b1a9f42a402d92f65fd4971f2e81/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:55:19.406421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a218d-5383-465b-b10b-75b0cddcdcea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:19.410378 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803a218d-5383-465b-b10b-75b0cddcdcea service swift] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:19.433657 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1143/2267] 10.222.0.71 () {60 vars in 1384 bytes} [Fri Aug 26 08:55:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 703 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 08:55:21.376148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45afc521-1896-4eb0-92d6-bde8982c8bec admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:21.562046 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45afc521-1896-4eb0-92d6-bde8982c8bec admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:21.571804 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99fde8-8563-4177-be11-abdc61056453 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:21.573367 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99fde8-8563-4177-be11-abdc61056453 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:21.574837 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99fde8-8563-4177-be11-abdc61056453 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:21.576029 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99fde8-8563-4177-be11-abdc61056453 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:21.581492 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4c99fde8-8563-4177-be11-abdc61056453 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:55:21.758292 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:55:21.828598 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99fde8-8563-4177-be11-abdc61056453 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:21.836756 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99fde8-8563-4177-be11-abdc61056453 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:21.875285 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1125/2268] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:55:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:55:29.418809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=69 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:55:31.861039 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=11 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:55:34.992959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab07a6-ef4a-4dfe-b196-09a441b30b62 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:34.993837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab07a6-ef4a-4dfe-b196-09a441b30b62 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:34.993837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab07a6-ef4a-4dfe-b196-09a441b30b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:35.029499 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-8eab07a6-ef4a-4dfe-b196-09a441b30b62 None None] MFA Rules not processed for user `ef15b1a9f42a402d92f65fd4971f2e81`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:55:35.030778 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8eab07a6-ef4a-4dfe-b196-09a441b30b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:55:35.105901 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1144/2269] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:55:34 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:55:35.119113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99fde8-8563-4177-be11-abdc61056453 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:35.121257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4c99fde8-8563-4177-be11-abdc61056453 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:55:35.206176 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99fde8-8563-4177-be11-abdc61056453 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:35.209675 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7057e-ec2b-457d-8b70-a12251606d14 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d178c717a3e49319545e70cb58cbbed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:35.210327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7057e-ec2b-457d-8b70-a12251606d14 service glance] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:35.210609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7057e-ec2b-457d-8b70-a12251606d14 service glance] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:35.211213 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7057e-ec2b-457d-8b70-a12251606d14 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:35.322253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc7057e-ec2b-457d-8b70-a12251606d14 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:35.326591 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc7057e-ec2b-457d-8b70-a12251606d14 service glance] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:35.341669 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1126/2270] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:55:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:55:35.552682 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbd40a-266d-4c5f-81c1-70d8b2329f0b None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:35.552980 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbd40a-266d-4c5f-81c1-70d8b2329f0b None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:35.553236 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cbd40a-266d-4c5f-81c1-70d8b2329f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:35.585997 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-21cbd40a-266d-4c5f-81c1-70d8b2329f0b None None] MFA Rules not processed for user `df07cffd82c74621855c8b8d7bfa989d`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:55:35.585997 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-21cbd40a-266d-4c5f-81c1-70d8b2329f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:55:35.679274 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1145/2271] 10.222.0.71 () {58 vars in 1009 bytes} [Fri Aug 26 08:55:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:55:35.716125 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7057e-ec2b-457d-8b70-a12251606d14 service glance] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:35.926983 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7057e-ec2b-457d-8b70-a12251606d14 service glance] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:35.941668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d525b06-4e27-4c09-beed-93ba8955a7aa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1723e961375b45b7b3bd99ddeb69683f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:35.943828 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d525b06-4e27-4c09-beed-93ba8955a7aa service swift] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:35.944173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d525b06-4e27-4c09-beed-93ba8955a7aa service swift] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:35.944456 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d525b06-4e27-4c09-beed-93ba8955a7aa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:35.956148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d525b06-4e27-4c09-beed-93ba8955a7aa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:35.958084 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d525b06-4e27-4c09-beed-93ba8955a7aa service swift] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:35.968407 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1127/2272] 10.222.0.71 () {60 vars in 1384 bytes} [Fri Aug 26 08:55:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 08:55:42.780765 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3077253-0a33-4878-9369-e3fec1225ad2 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:42.781586 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3077253-0a33-4878-9369-e3fec1225ad2 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:42.785302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3077253-0a33-4878-9369-e3fec1225ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:42.828081 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-c3077253-0a33-4878-9369-e3fec1225ad2 None None] MFA Rules not processed for user `406be06d6adf40bbb48d9baf12022988`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:55:42.830068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c3077253-0a33-4878-9369-e3fec1225ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:55:42.989415 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1146/2273] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:55:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:55:45.963837 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=80 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:55:50.234991 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef03b87-7269-489a-9bba-e166a8416118 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:50.234991 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef03b87-7269-489a-9bba-e166a8416118 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:50.234991 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef03b87-7269-489a-9bba-e166a8416118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:50.258286 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-3ef03b87-7269-489a-9bba-e166a8416118 None None] MFA Rules not processed for user `2cee3394dbdd4cb3a52005ee357d3be7`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:55:50.260134 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3ef03b87-7269-489a-9bba-e166a8416118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:55:50.267591 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1128/2274] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:55:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:55:52.945917 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=84 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:55:54.315187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f520c-09a4-42ae-a5f1-0b49469916d6 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:54.316260 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f520c-09a4-42ae-a5f1-0b49469916d6 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:54.316260 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f520c-09a4-42ae-a5f1-0b49469916d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:54.342306 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-b55f520c-09a4-42ae-a5f1-0b49469916d6 None None] MFA Rules not processed for user `df07cffd82c74621855c8b8d7bfa989d`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:55:54.343925 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b55f520c-09a4-42ae-a5f1-0b49469916d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:55:54.354253 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1147/2275] 10.222.0.71 () {58 vars in 967 bytes} [Fri Aug 26 08:55:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:55:54.375686 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef03b87-7269-489a-9bba-e166a8416118 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:54.395790 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef03b87-7269-489a-9bba-e166a8416118 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:54.402588 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86f7b8-66f7-4783-b57b-7eb207853987 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1723e961375b45b7b3bd99ddeb69683f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:54.403418 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86f7b8-66f7-4783-b57b-7eb207853987 service swift] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:54.403734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86f7b8-66f7-4783-b57b-7eb207853987 service swift] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:54.404248 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86f7b8-66f7-4783-b57b-7eb207853987 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:54.415581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc86f7b8-66f7-4783-b57b-7eb207853987 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:54.417975 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc86f7b8-66f7-4783-b57b-7eb207853987 service swift] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:54.434406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1129/2276] 10.222.0.71 () {60 vars in 1384 bytes} [Fri Aug 26 08:55:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 08:55:56.959376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fa3f0-2e12-448d-b4fd-42f6e4196d89 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:56.960527 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fa3f0-2e12-448d-b4fd-42f6e4196d89 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:56.961015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fa3f0-2e12-448d-b4fd-42f6e4196d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:56.995973 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-606fa3f0-2e12-448d-b4fd-42f6e4196d89 None None] MFA Rules not processed for user `df07cffd82c74621855c8b8d7bfa989d`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:55:56.998327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-606fa3f0-2e12-448d-b4fd-42f6e4196d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:55:57.013749 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1148/2277] 10.222.0.71 () {58 vars in 1009 bytes} [Fri Aug 26 08:55:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:55:57.041995 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86f7b8-66f7-4783-b57b-7eb207853987 service swift] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:57.084836 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86f7b8-66f7-4783-b57b-7eb207853987 service swift] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:57.096345 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06af1a8-6f5b-4194-b8ab-bafa5b91502f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1723e961375b45b7b3bd99ddeb69683f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:57.098039 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06af1a8-6f5b-4194-b8ab-bafa5b91502f service swift] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:57.098039 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06af1a8-6f5b-4194-b8ab-bafa5b91502f service swift] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:57.098273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06af1a8-6f5b-4194-b8ab-bafa5b91502f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:57.112805 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06af1a8-6f5b-4194-b8ab-bafa5b91502f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:57.115603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06af1a8-6f5b-4194-b8ab-bafa5b91502f service swift] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:57.129179 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1130/2278] 10.222.0.71 () {60 vars in 1384 bytes} [Fri Aug 26 08:55:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 08:55:57.979754 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08e8b9-e0e7-494a-9d0a-f96812a8a53f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:57.980362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08e8b9-e0e7-494a-9d0a-f96812a8a53f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:57.980362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08e8b9-e0e7-494a-9d0a-f96812a8a53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:58.013237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-4e08e8b9-e0e7-494a-9d0a-f96812a8a53f None None] MFA Rules not processed for user `df07cffd82c74621855c8b8d7bfa989d`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:55:58.015450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4e08e8b9-e0e7-494a-9d0a-f96812a8a53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:55:58.028897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1149/2279] 10.222.0.71 () {58 vars in 967 bytes} [Fri Aug 26 08:55:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:55:58.049676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06af1a8-6f5b-4194-b8ab-bafa5b91502f service swift] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:58.061532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06af1a8-6f5b-4194-b8ab-bafa5b91502f service swift] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:58.066800 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e7edc-05de-43d4-adb8-514b6595fdfe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1723e961375b45b7b3bd99ddeb69683f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:58.067594 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e7edc-05de-43d4-adb8-514b6595fdfe service swift] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:58.067881 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e7edc-05de-43d4-adb8-514b6595fdfe service swift] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:58.068146 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e7edc-05de-43d4-adb8-514b6595fdfe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:58.077776 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648e7edc-05de-43d4-adb8-514b6595fdfe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:58.079866 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648e7edc-05de-43d4-adb8-514b6595fdfe service swift] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:58.089466 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1131/2280] 10.222.0.71 () {60 vars in 1384 bytes} [Fri Aug 26 08:55:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 08:55:58.379121 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee1e9e-7423-4a57-9c57-5f02a793e1fe None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:58.379620 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee1e9e-7423-4a57-9c57-5f02a793e1fe None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:58.380118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee1e9e-7423-4a57-9c57-5f02a793e1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:58.411574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-17ee1e9e-7423-4a57-9c57-5f02a793e1fe None None] MFA Rules not processed for user `df07cffd82c74621855c8b8d7bfa989d`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:55:58.413503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-17ee1e9e-7423-4a57-9c57-5f02a793e1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:55:58.424939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1150/2281] 10.222.0.71 () {58 vars in 1009 bytes} [Fri Aug 26 08:55:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:55:58.440084 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e7edc-05de-43d4-adb8-514b6595fdfe service swift] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:58.451743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e7edc-05de-43d4-adb8-514b6595fdfe service swift] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:58.458341 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2defd2e6-267f-4c97-9e5e-1ea5e53737d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1723e961375b45b7b3bd99ddeb69683f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:58.458877 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defd2e6-267f-4c97-9e5e-1ea5e53737d8 service swift] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:58.459180 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defd2e6-267f-4c97-9e5e-1ea5e53737d8 service swift] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:58.459446 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defd2e6-267f-4c97-9e5e-1ea5e53737d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:58.469045 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2defd2e6-267f-4c97-9e5e-1ea5e53737d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:58.470646 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2defd2e6-267f-4c97-9e5e-1ea5e53737d8 service swift] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:58.481045 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1132/2282] 10.222.0.71 () {60 vars in 1384 bytes} [Fri Aug 26 08:55:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 08:55:58.879654 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d4374-c183-4041-a3d0-448f7dae87eb None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:58.880078 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d4374-c183-4041-a3d0-448f7dae87eb None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:58.880542 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d4374-c183-4041-a3d0-448f7dae87eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:58.902688 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-444d4374-c183-4041-a3d0-448f7dae87eb None None] MFA Rules not processed for user `df07cffd82c74621855c8b8d7bfa989d`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:55:58.904288 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-444d4374-c183-4041-a3d0-448f7dae87eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:55:58.915491 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1151/2283] 10.222.0.71 () {58 vars in 967 bytes} [Fri Aug 26 08:55:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:55:58.929025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2defd2e6-267f-4c97-9e5e-1ea5e53737d8 service swift] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:58.942763 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2defd2e6-267f-4c97-9e5e-1ea5e53737d8 service swift] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:58.952172 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3468eee-bb01-4d3e-bc6a-1627993abc7c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1723e961375b45b7b3bd99ddeb69683f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:58.952829 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3468eee-bb01-4d3e-bc6a-1627993abc7c service swift] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:58.953109 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3468eee-bb01-4d3e-bc6a-1627993abc7c service swift] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:58.953381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3468eee-bb01-4d3e-bc6a-1627993abc7c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:58.964579 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3468eee-bb01-4d3e-bc6a-1627993abc7c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:58.966043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3468eee-bb01-4d3e-bc6a-1627993abc7c service swift] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:58.976215 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1133/2284] 10.222.0.71 () {60 vars in 1384 bytes} [Fri Aug 26 08:55:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 08:55:59.058329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a40c4-5111-44de-b123-fd227fb79ab7 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:59.059037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a40c4-5111-44de-b123-fd227fb79ab7 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:59.059037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a40c4-5111-44de-b123-fd227fb79ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:59.088956 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-366a40c4-5111-44de-b123-fd227fb79ab7 None None] MFA Rules not processed for user `df07cffd82c74621855c8b8d7bfa989d`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:55:59.091107 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-366a40c4-5111-44de-b123-fd227fb79ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:55:59.101365 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1152/2285] 10.222.0.71 () {58 vars in 1009 bytes} [Fri Aug 26 08:55:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:55:59.119308 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3468eee-bb01-4d3e-bc6a-1627993abc7c service swift] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:55:59.132556 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3468eee-bb01-4d3e-bc6a-1627993abc7c service swift] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:55:59.140757 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e474c-de10-43a6-89b0-abf0d81b75fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1723e961375b45b7b3bd99ddeb69683f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:55:59.141430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e474c-de10-43a6-89b0-abf0d81b75fe service swift] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:55:59.141716 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e474c-de10-43a6-89b0-abf0d81b75fe service swift] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:55:59.141991 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e474c-de10-43a6-89b0-abf0d81b75fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:55:59.163908 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e474c-de10-43a6-89b0-abf0d81b75fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:55:59.166549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e474c-de10-43a6-89b0-abf0d81b75fe service swift] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:55:59.181751 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1134/2286] 10.222.0.71 () {60 vars in 1384 bytes} [Fri Aug 26 08:55:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 08:56:02.141880 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a40c4-5111-44de-b123-fd227fb79ab7 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:56:02.146422 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-366a40c4-5111-44de-b123-fd227fb79ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:56:02.299849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a40c4-5111-44de-b123-fd227fb79ab7 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:56:02.308643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c7244-872c-4aab-9bc8-fab62630308d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:56:02.309931 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c7244-872c-4aab-9bc8-fab62630308d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:56:02.310496 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c7244-872c-4aab-9bc8-fab62630308d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:56:02.311087 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c7244-872c-4aab-9bc8-fab62630308d admin admin] PATH_INFO: `/v3/users/3d3ea95f39e24567af6f1be8f7e2a60b` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:56:02.313600 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c7244-872c-4aab-9bc8-fab62630308d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d3ea95f39e24567af6f1be8f7e2a60b)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:56:02.319285 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c7244-872c-4aab-9bc8-fab62630308d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:56:02.411089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-521c7244-872c-4aab-9bc8-fab62630308d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d3ea95f39e24567af6f1be8f7e2a60b'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:56:02.429456 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-521c7244-872c-4aab-9bc8-fab62630308d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d3ea95f39e24567af6f1be8f7e2a60b'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:56:02.449494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-521c7244-872c-4aab-9bc8-fab62630308d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d3ea95f39e24567af6f1be8f7e2a60b'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:56:02.469320 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-521c7244-872c-4aab-9bc8-fab62630308d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d3ea95f39e24567af6f1be8f7e2a60b'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:56:02.470551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-521c7244-872c-4aab-9bc8-fab62630308d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d3ea95f39e24567af6f1be8f7e2a60b'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:56:02.484404 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1153/2287] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:56:02 2022] DELETE /identity/v3/users/3d3ea95f39e24567af6f1be8f7e2a60b => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:56:02.492388 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2773b-5935-40d4-a85f-f9e378342a15 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:56:02.493115 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2773b-5935-40d4-a85f-f9e378342a15 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:56:02.493115 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2773b-5935-40d4-a85f-f9e378342a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:56:02.519234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-d4b2773b-5935-40d4-a85f-f9e378342a15 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:56:02.520618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d4b2773b-5935-40d4-a85f-f9e378342a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:56:02.625420 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1135/2288] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:56:02.637135 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c7244-872c-4aab-9bc8-fab62630308d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:56:02.638987 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-521c7244-872c-4aab-9bc8-fab62630308d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:56:02.755108 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:56:02.777830 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c7244-872c-4aab-9bc8-fab62630308d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:56:02.798237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e042095-8593-436b-b782-949bc88ff971 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:56:02.799053 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e042095-8593-436b-b782-949bc88ff971 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:56:02.799435 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e042095-8593-436b-b782-949bc88ff971 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:56:02.799777 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e042095-8593-436b-b782-949bc88ff971 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:56:02.817904 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e042095-8593-436b-b782-949bc88ff971 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:56:02.819635 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e042095-8593-436b-b782-949bc88ff971 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:56:02.839774 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1154/2289] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:56:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:56:03.133312 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2773b-5935-40d4-a85f-f9e378342a15 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:56:03.135969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d4b2773b-5935-40d4-a85f-f9e378342a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:56:03.153005 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2773b-5935-40d4-a85f-f9e378342a15 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:56:03.159352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920064a-c06f-414c-ad2d-759104d6f12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:56:03.160442 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920064a-c06f-414c-ad2d-759104d6f12b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:56:03.161025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920064a-c06f-414c-ad2d-759104d6f12b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:56:03.161699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920064a-c06f-414c-ad2d-759104d6f12b admin admin] PATH_INFO: `/v3/projects/d9c2937d8e1243c185dab8e97f0a6e90` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:56:03.163994 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920064a-c06f-414c-ad2d-759104d6f12b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9c2937d8e1243c185dab8e97f0a6e90)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:56:03.168135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920064a-c06f-414c-ad2d-759104d6f12b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:56:03.231510 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5920064a-c06f-414c-ad2d-759104d6f12b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9c2937d8e1243c185dab8e97f0a6e90', 'request_id': 'req-5920064a-c06f-414c-ad2d-759104d6f12b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:56:03.246918 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5920064a-c06f-414c-ad2d-759104d6f12b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9c2937d8e1243c185dab8e97f0a6e90', 'request_id': 'req-5920064a-c06f-414c-ad2d-759104d6f12b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:56:03.262115 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5920064a-c06f-414c-ad2d-759104d6f12b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9c2937d8e1243c185dab8e97f0a6e90', 'request_id': 'req-5920064a-c06f-414c-ad2d-759104d6f12b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:56:03.272109 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:56:03.278242 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5920064a-c06f-414c-ad2d-759104d6f12b admin admin] The token cache is being invalidate because project d9c2937d8e1243c185dab8e97f0a6e90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:56:03.278560 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5920064a-c06f-414c-ad2d-759104d6f12b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:56:03.279727 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5920064a-c06f-414c-ad2d-759104d6f12b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:56:03.282620 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1136/2290] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:56:03 2022] DELETE /identity/v3/projects/d9c2937d8e1243c185dab8e97f0a6e90 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:56:12.833551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=8 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:56:13.284036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:56:30.468910 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e042095-8593-436b-b782-949bc88ff971 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:56:30.470844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2e042095-8593-436b-b782-949bc88ff971 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:56:30.482520 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e042095-8593-436b-b782-949bc88ff971 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:56:30.542968 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe9e9e-1f69-4350-aef7-f5fd1d25505b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:56:30.543594 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe9e9e-1f69-4350-aef7-f5fd1d25505b service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:56:30.543869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe9e9e-1f69-4350-aef7-f5fd1d25505b service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:56:30.544142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fe9e9e-1f69-4350-aef7-f5fd1d25505b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:56:30.545941 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e2fe9e9e-1f69-4350-aef7-f5fd1d25505b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:56:30.639622 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe9e9e-1f69-4350-aef7-f5fd1d25505b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:56:30.642014 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fe9e9e-1f69-4350-aef7-f5fd1d25505b service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:56:30.655202 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1155/2291] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:56:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:56:34.255944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920064a-c06f-414c-ad2d-759104d6f12b admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:56:34.257823 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5920064a-c06f-414c-ad2d-759104d6f12b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:56:34.272276 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920064a-c06f-414c-ad2d-759104d6f12b admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:56:34.276968 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858f878-9fa6-4ab7-8e4f-2ba381f1ce56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:56:34.277637 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858f878-9fa6-4ab7-8e4f-2ba381f1ce56 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:56:34.278051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858f878-9fa6-4ab7-8e4f-2ba381f1ce56 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:56:34.278242 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858f878-9fa6-4ab7-8e4f-2ba381f1ce56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:56:34.280222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1858f878-9fa6-4ab7-8e4f-2ba381f1ce56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:56:34.298670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858f878-9fa6-4ab7-8e4f-2ba381f1ce56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:56:34.301444 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858f878-9fa6-4ab7-8e4f-2ba381f1ce56 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:56:34.316583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1137/2292] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:56:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:56:40.649063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=54 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:56:41.016989 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe9e9e-1f69-4350-aef7-f5fd1d25505b service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:56:41.028071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fe9e9e-1f69-4350-aef7-f5fd1d25505b service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:56:41.032650 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d12de-5a12-4db1-b943-6f6501f17a26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:56:41.033291 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d12de-5a12-4db1-b943-6f6501f17a26 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:56:41.033571 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d12de-5a12-4db1-b943-6f6501f17a26 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:56:41.033837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d12de-5a12-4db1-b943-6f6501f17a26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:56:41.035902 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d54d12de-5a12-4db1-b943-6f6501f17a26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:56:41.106299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d12de-5a12-4db1-b943-6f6501f17a26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:56:41.107985 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d12de-5a12-4db1-b943-6f6501f17a26 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:56:41.118543 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1156/2293] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:56:41 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:56:44.310742 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=6 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:56:51.113943 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=30 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:57:05.066706 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858f878-9fa6-4ab7-8e4f-2ba381f1ce56 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:57:05.070256 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1858f878-9fa6-4ab7-8e4f-2ba381f1ce56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:57:05.085351 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858f878-9fa6-4ab7-8e4f-2ba381f1ce56 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:57:05.091835 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7850f48-ecda-4dcc-abd2-343971bd90c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:57:05.092415 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7850f48-ecda-4dcc-abd2-343971bd90c2 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:57:05.092690 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7850f48-ecda-4dcc-abd2-343971bd90c2 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:57:05.092954 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7850f48-ecda-4dcc-abd2-343971bd90c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:57:05.094861 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b7850f48-ecda-4dcc-abd2-343971bd90c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:57:05.162445 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7850f48-ecda-4dcc-abd2-343971bd90c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:57:05.164255 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7850f48-ecda-4dcc-abd2-343971bd90c2 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:57:05.173760 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1138/2294] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:57:05 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:57:08.626019 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d12de-5a12-4db1-b943-6f6501f17a26 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:57:08.627941 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d54d12de-5a12-4db1-b943-6f6501f17a26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:57:08.644425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d12de-5a12-4db1-b943-6f6501f17a26 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:57:08.651402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682e33f-c6e2-44fc-8987-7c72bfa31adc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:57:08.652023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682e33f-c6e2-44fc-8987-7c72bfa31adc service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:57:08.652295 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682e33f-c6e2-44fc-8987-7c72bfa31adc service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:57:08.652560 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682e33f-c6e2-44fc-8987-7c72bfa31adc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:57:08.654645 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a682e33f-c6e2-44fc-8987-7c72bfa31adc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:57:08.721193 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a682e33f-c6e2-44fc-8987-7c72bfa31adc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:57:08.722946 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a682e33f-c6e2-44fc-8987-7c72bfa31adc service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:57:08.732865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1157/2295] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:57:08 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:57:15.169396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=28 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:57:18.728465 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=28 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:57:22.873327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045317c4-a47e-40c9-83ae-c4baa9f00439 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:57:22.874031 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045317c4-a47e-40c9-83ae-c4baa9f00439 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:57:22.874031 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045317c4-a47e-40c9-83ae-c4baa9f00439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:57:22.894473 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-045317c4-a47e-40c9-83ae-c4baa9f00439 None None] MFA Rules not processed for user `63e1c5a24be04ca4abd5d5109c5763f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:57:22.895768 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-045317c4-a47e-40c9-83ae-c4baa9f00439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:57:22.903774 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1139/2296] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:57:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:57:23.264603 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682e33f-c6e2-44fc-8987-7c72bfa31adc service nova] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:57:23.266143 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a682e33f-c6e2-44fc-8987-7c72bfa31adc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:57:23.281619 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682e33f-c6e2-44fc-8987-7c72bfa31adc service nova] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:57:23.287379 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb240a-0279-4692-ad66-86a8ec5ad9d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:57:23.288422 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb240a-0279-4692-ad66-86a8ec5ad9d4 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:57:23.288913 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb240a-0279-4692-ad66-86a8ec5ad9d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:57:23.289251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb240a-0279-4692-ad66-86a8ec5ad9d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:57:23.291580 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2cdb240a-0279-4692-ad66-86a8ec5ad9d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:57:23.302613 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdb240a-0279-4692-ad66-86a8ec5ad9d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:57:23.304117 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdb240a-0279-4692-ad66-86a8ec5ad9d4 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:57:23.313991 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1158/2297] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:57:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:57:23.689452 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045317c4-a47e-40c9-83ae-c4baa9f00439 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:57:23.691122 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-045317c4-a47e-40c9-83ae-c4baa9f00439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:57:23.776245 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045317c4-a47e-40c9-83ae-c4baa9f00439 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:57:23.783134 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ad5da-a238-4936-8a54-22b825b45075 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf5e2b4f5e440a7942adea3d51cec75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:57:23.783666 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ad5da-a238-4936-8a54-22b825b45075 service placement] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:57:23.783942 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ad5da-a238-4936-8a54-22b825b45075 service placement] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:57:23.784219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ad5da-a238-4936-8a54-22b825b45075 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:57:23.786148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0f5ad5da-a238-4936-8a54-22b825b45075 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:57:23.799423 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ad5da-a238-4936-8a54-22b825b45075 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:57:23.801114 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ad5da-a238-4936-8a54-22b825b45075 service placement] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:57:23.812090 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1140/2298] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:57:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:57:26.320896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb240a-0279-4692-ad66-86a8ec5ad9d4 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:57:26.331598 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb240a-0279-4692-ad66-86a8ec5ad9d4 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:57:26.335810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ad3a6-95e3-4f53-9bf2-6ef96562b6a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:57:26.336532 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ad3a6-95e3-4f53-9bf2-6ef96562b6a8 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:57:26.336877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ad3a6-95e3-4f53-9bf2-6ef96562b6a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:57:26.337216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ad3a6-95e3-4f53-9bf2-6ef96562b6a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:57:26.339637 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2f2ad3a6-95e3-4f53-9bf2-6ef96562b6a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:57:26.350744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ad3a6-95e3-4f53-9bf2-6ef96562b6a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:57:26.352261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ad3a6-95e3-4f53-9bf2-6ef96562b6a8 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:57:26.364581 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1159/2299] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:57:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:57:27.091007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ad5da-a238-4936-8a54-22b825b45075 service placement] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:57:27.104207 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ad5da-a238-4936-8a54-22b825b45075 service placement] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:57:27.111035 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c9af5-8104-4392-bac1-b603829c4a07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:57:27.111671 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c9af5-8104-4392-bac1-b603829c4a07 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:57:27.111957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c9af5-8104-4392-bac1-b603829c4a07 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:57:27.112232 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c9af5-8104-4392-bac1-b603829c4a07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:57:27.114245 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d68c9af5-8104-4392-bac1-b603829c4a07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:57:27.130370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68c9af5-8104-4392-bac1-b603829c4a07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:57:27.132380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68c9af5-8104-4392-bac1-b603829c4a07 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:57:27.163316 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1141/2300] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:57:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:57:33.299177 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ad3a6-95e3-4f53-9bf2-6ef96562b6a8 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:57:33.318443 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ad3a6-95e3-4f53-9bf2-6ef96562b6a8 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:57:33.326266 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195bc95-500d-4a62-93aa-1510385194ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:57:33.327168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195bc95-500d-4a62-93aa-1510385194ee service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:57:33.327571 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195bc95-500d-4a62-93aa-1510385194ee service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:57:33.327954 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195bc95-500d-4a62-93aa-1510385194ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:57:33.330153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b195bc95-500d-4a62-93aa-1510385194ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:57:33.348040 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195bc95-500d-4a62-93aa-1510385194ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:57:33.349789 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195bc95-500d-4a62-93aa-1510385194ee service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:57:33.363994 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1160/2301] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:57:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:57:37.142337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=41 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:57:38.012014 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c9af5-8104-4392-bac1-b603829c4a07 service nova] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:57:38.024392 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c9af5-8104-4392-bac1-b603829c4a07 service nova] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:57:38.030921 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35af3f-9ef7-4fd8-ad5b-362dc6ebc7c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:57:38.031555 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35af3f-9ef7-4fd8-ad5b-362dc6ebc7c6 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:57:38.031833 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35af3f-9ef7-4fd8-ad5b-362dc6ebc7c6 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:57:38.032105 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35af3f-9ef7-4fd8-ad5b-362dc6ebc7c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:57:38.034105 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1a35af3f-9ef7-4fd8-ad5b-362dc6ebc7c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:57:38.054679 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a35af3f-9ef7-4fd8-ad5b-362dc6ebc7c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:57:38.056509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a35af3f-9ef7-4fd8-ad5b-362dc6ebc7c6 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:57:38.070856 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1142/2302] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:57:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:57:40.457110 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f42b5-62d1-4a48-86c5-738d57505de8 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:57:40.458066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f42b5-62d1-4a48-86c5-738d57505de8 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:57:40.458503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f42b5-62d1-4a48-86c5-738d57505de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:57:40.483416 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ca0f42b5-62d1-4a48-86c5-738d57505de8 None None] MFA Rules not processed for user `3aced0ed17b946f9961d87a1d4446c37`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:57:40.484963 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ca0f42b5-62d1-4a48-86c5-738d57505de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:57:40.497024 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1161/2303] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:57:40 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:57:40.963963 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35af3f-9ef7-4fd8-ad5b-362dc6ebc7c6 service nova] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:57:40.966239 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1a35af3f-9ef7-4fd8-ad5b-362dc6ebc7c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:57:41.067308 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35af3f-9ef7-4fd8-ad5b-362dc6ebc7c6 service nova] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:57:41.070996 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e64ce4-d80f-44fb-85a5-1ffcdecb2bb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:57:41.071697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e64ce4-d80f-44fb-85a5-1ffcdecb2bb1 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:57:41.072207 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e64ce4-d80f-44fb-85a5-1ffcdecb2bb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:57:41.073008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e64ce4-d80f-44fb-85a5-1ffcdecb2bb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:57:41.075189 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-92e64ce4-d80f-44fb-85a5-1ffcdecb2bb1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:57:41.148603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e64ce4-d80f-44fb-85a5-1ffcdecb2bb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:57:41.150346 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e64ce4-d80f-44fb-85a5-1ffcdecb2bb1 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:57:41.187775 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1143/2304] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:57:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:57:50.474809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=21 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:57:51.160416 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=54 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:57:54.830801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff07db0-1b91-4268-af29-2440554e8e83 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:57:54.832456 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff07db0-1b91-4268-af29-2440554e8e83 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:57:54.832456 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff07db0-1b91-4268-af29-2440554e8e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:57:54.853572 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-1ff07db0-1b91-4268-af29-2440554e8e83 None None] MFA Rules not processed for user `b6f722499abc409a8a345bcb70114eaf`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:57:54.854835 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1ff07db0-1b91-4268-af29-2440554e8e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:57:54.862139 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1162/2305] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:57:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:57:57.384738 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975136af-2283-4c3e-9103-7fa76aee4977 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:57:57.385670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975136af-2283-4c3e-9103-7fa76aee4977 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:57:57.387437 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975136af-2283-4c3e-9103-7fa76aee4977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:57:57.414783 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-975136af-2283-4c3e-9103-7fa76aee4977 None None] MFA Rules not processed for user `406be06d6adf40bbb48d9baf12022988`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:57:57.416192 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-975136af-2283-4c3e-9103-7fa76aee4977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:57:57.512834 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1144/2306] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:57:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:57:57.978263 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff07db0-1b91-4268-af29-2440554e8e83 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:57:57.990460 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff07db0-1b91-4268-af29-2440554e8e83 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:57:57.996086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a181a8f-c4a7-4956-8c8c-a8a2621f33d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:57:57.997192 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a181a8f-c4a7-4956-8c8c-a8a2621f33d9 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:57:57.997558 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a181a8f-c4a7-4956-8c8c-a8a2621f33d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:57:57.997910 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a181a8f-c4a7-4956-8c8c-a8a2621f33d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:57:58.001694 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1a181a8f-c4a7-4956-8c8c-a8a2621f33d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:57:58.021348 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a181a8f-c4a7-4956-8c8c-a8a2621f33d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:57:58.022701 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a181a8f-c4a7-4956-8c8c-a8a2621f33d9 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:57:58.037651 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1163/2307] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:57:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:58:07.498071 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=25 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:58:08.031674 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=9 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:59:09.380104 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975136af-2283-4c3e-9103-7fa76aee4977 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:09.391179 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975136af-2283-4c3e-9103-7fa76aee4977 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:09.396432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8c393-6946-4e1f-9703-ecec8940cbc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:09.397086 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8c393-6946-4e1f-9703-ecec8940cbc9 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:09.397359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8c393-6946-4e1f-9703-ecec8940cbc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:09.397665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8c393-6946-4e1f-9703-ecec8940cbc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:09.400041 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-dea8c393-6946-4e1f-9703-ecec8940cbc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:09.476218 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8c393-6946-4e1f-9703-ecec8940cbc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:09.478923 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8c393-6946-4e1f-9703-ecec8940cbc9 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:09.492826 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1145/2308] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:59:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:59:10.451947 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6da56-a874-43dc-9ab9-3a51397c0cff None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:10.452906 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6da56-a874-43dc-9ab9-3a51397c0cff None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:10.452906 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6da56-a874-43dc-9ab9-3a51397c0cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:10.477110 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-36c6da56-a874-43dc-9ab9-3a51397c0cff None None] MFA Rules not processed for user `df07cffd82c74621855c8b8d7bfa989d`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:59:10.479699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-36c6da56-a874-43dc-9ab9-3a51397c0cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:10.563542 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1164/2309] 10.222.0.71 () {58 vars in 1009 bytes} [Fri Aug 26 08:59:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:59:10.618852 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8c393-6946-4e1f-9703-ecec8940cbc9 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:10.620559 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-dea8c393-6946-4e1f-9703-ecec8940cbc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:10.715183 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8c393-6946-4e1f-9703-ecec8940cbc9 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:10.724466 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a92923-8d2c-4d41-96b3-dc89f706d0c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1723e961375b45b7b3bd99ddeb69683f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:10.725474 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a92923-8d2c-4d41-96b3-dc89f706d0c8 service swift] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:10.725943 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a92923-8d2c-4d41-96b3-dc89f706d0c8 service swift] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:10.726392 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a92923-8d2c-4d41-96b3-dc89f706d0c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:10.738377 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a92923-8d2c-4d41-96b3-dc89f706d0c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:10.740047 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a92923-8d2c-4d41-96b3-dc89f706d0c8 service swift] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:10.750278 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1146/2310] 10.222.0.71 () {60 vars in 1384 bytes} [Fri Aug 26 08:59:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 08:59:14.762040 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c6da56-a874-43dc-9ab9-3a51397c0cff None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:14.794432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c6da56-a874-43dc-9ab9-3a51397c0cff None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:14.803160 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e8c6d-32af-4bdf-92a1-d3fae4b139e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:14.804068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e8c6d-32af-4bdf-92a1-d3fae4b139e1 service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:14.804581 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e8c6d-32af-4bdf-92a1-d3fae4b139e1 service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:14.805027 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e8c6d-32af-4bdf-92a1-d3fae4b139e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:14.807088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-154e8c6d-32af-4bdf-92a1-d3fae4b139e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:14.822541 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e8c6d-32af-4bdf-92a1-d3fae4b139e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:14.824697 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e8c6d-32af-4bdf-92a1-d3fae4b139e1 service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:14.853542 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1165/2311] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:59:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:59:16.654385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a92923-8d2c-4d41-96b3-dc89f706d0c8 service swift] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:16.656900 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-28a92923-8d2c-4d41-96b3-dc89f706d0c8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:16.686855 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a92923-8d2c-4d41-96b3-dc89f706d0c8 service swift] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:16.710268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479502b0-cf90-434a-ac2c-df7684646d8a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf5e2b4f5e440a7942adea3d51cec75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:16.710914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479502b0-cf90-434a-ac2c-df7684646d8a service placement] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:16.711607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479502b0-cf90-434a-ac2c-df7684646d8a service placement] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:16.711880 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479502b0-cf90-434a-ac2c-df7684646d8a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:16.713689 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-479502b0-cf90-434a-ac2c-df7684646d8a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:16.727498 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479502b0-cf90-434a-ac2c-df7684646d8a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:16.730237 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479502b0-cf90-434a-ac2c-df7684646d8a service placement] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:16.743688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1147/2312] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:59:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:59:16.767977 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e8c6d-32af-4bdf-92a1-d3fae4b139e1 service nova] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:16.769593 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-154e8c6d-32af-4bdf-92a1-d3fae4b139e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:16.793016 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e8c6d-32af-4bdf-92a1-d3fae4b139e1 service nova] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:16.798980 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eca0e4-cd7b-4fe0-8895-98ec49cf70cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:16.800531 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eca0e4-cd7b-4fe0-8895-98ec49cf70cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:16.801033 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eca0e4-cd7b-4fe0-8895-98ec49cf70cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:16.801874 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eca0e4-cd7b-4fe0-8895-98ec49cf70cd admin admin] PATH_INFO: `/v3/users/3aced0ed17b946f9961d87a1d4446c37` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:16.804419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eca0e4-cd7b-4fe0-8895-98ec49cf70cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aced0ed17b946f9961d87a1d4446c37)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:16.807253 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eca0e4-cd7b-4fe0-8895-98ec49cf70cd admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:16.956150 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f4eca0e4-cd7b-4fe0-8895-98ec49cf70cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aced0ed17b946f9961d87a1d4446c37'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:16.988346 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f4eca0e4-cd7b-4fe0-8895-98ec49cf70cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aced0ed17b946f9961d87a1d4446c37'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:16.999300 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f4eca0e4-cd7b-4fe0-8895-98ec49cf70cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aced0ed17b946f9961d87a1d4446c37'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:17.024696 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f4eca0e4-cd7b-4fe0-8895-98ec49cf70cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aced0ed17b946f9961d87a1d4446c37'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:17.026007 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f4eca0e4-cd7b-4fe0-8895-98ec49cf70cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aced0ed17b946f9961d87a1d4446c37'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:17.045507 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1166/2313] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:59:16 2022] DELETE /identity/v3/users/3aced0ed17b946f9961d87a1d4446c37 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:59:17.331777 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479502b0-cf90-434a-ac2c-df7684646d8a service placement] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:17.367821 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479502b0-cf90-434a-ac2c-df7684646d8a service placement] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:17.537922 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470de78-369d-458b-962e-a21bd7c5b3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:17.543119 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470de78-369d-458b-962e-a21bd7c5b3b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:17.545853 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470de78-369d-458b-962e-a21bd7c5b3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:17.546727 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470de78-369d-458b-962e-a21bd7c5b3b6 admin admin] PATH_INFO: `/v3/projects/01d9fa7bc7124476aa295e69b120becd` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:17.550257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470de78-369d-458b-962e-a21bd7c5b3b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01d9fa7bc7124476aa295e69b120becd)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:17.552765 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470de78-369d-458b-962e-a21bd7c5b3b6 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:17.580872 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:59:17.668678 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e470de78-369d-458b-962e-a21bd7c5b3b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01d9fa7bc7124476aa295e69b120becd', 'request_id': 'req-e470de78-369d-458b-962e-a21bd7c5b3b6'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:17.683475 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e470de78-369d-458b-962e-a21bd7c5b3b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01d9fa7bc7124476aa295e69b120becd', 'request_id': 'req-e470de78-369d-458b-962e-a21bd7c5b3b6'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:17.693198 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e470de78-369d-458b-962e-a21bd7c5b3b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01d9fa7bc7124476aa295e69b120becd', 'request_id': 'req-e470de78-369d-458b-962e-a21bd7c5b3b6'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:17.708882 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e470de78-369d-458b-962e-a21bd7c5b3b6 admin admin] The token cache is being invalidate because project 01d9fa7bc7124476aa295e69b120becd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:59:17.709489 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e470de78-369d-458b-962e-a21bd7c5b3b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:17.710172 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e470de78-369d-458b-962e-a21bd7c5b3b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:17.712121 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1148/2314] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:59:17 2022] DELETE /identity/v3/projects/01d9fa7bc7124476aa295e69b120becd => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:59:17.722456 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d9587-8a69-40b6-95ff-92700f71d673 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:17.722740 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d9587-8a69-40b6-95ff-92700f71d673 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:17.723058 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d9587-8a69-40b6-95ff-92700f71d673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:17.754023 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-277d9587-8a69-40b6-95ff-92700f71d673 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:59:17.755894 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-277d9587-8a69-40b6-95ff-92700f71d673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:17.875021 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1167/2315] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:59:17.883738 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237fce2-56d0-4b71-9fd8-b84b0e4e5608 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:17.884168 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237fce2-56d0-4b71-9fd8-b84b0e4e5608 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:17.884586 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237fce2-56d0-4b71-9fd8-b84b0e4e5608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:17.918060 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-4237fce2-56d0-4b71-9fd8-b84b0e4e5608 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:59:17.920188 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4237fce2-56d0-4b71-9fd8-b84b0e4e5608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:17.940769 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1149/2316] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:59:17.946815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d9587-8a69-40b6-95ff-92700f71d673 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:17.969466 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:59:17.986728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d9587-8a69-40b6-95ff-92700f71d673 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:17.999830 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2ae0e-7ec7-4872-a472-7fad34827c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:18.001597 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2ae0e-7ec7-4872-a472-7fad34827c7a admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:18.002212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2ae0e-7ec7-4872-a472-7fad34827c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:18.002748 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2ae0e-7ec7-4872-a472-7fad34827c7a admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:18.004242 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f2ae0e-7ec7-4872-a472-7fad34827c7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:59:18.004826 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f2ae0e-7ec7-4872-a472-7fad34827c7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:18.008870 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f2ae0e-7ec7-4872-a472-7fad34827c7a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:18.040240 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1168/2317] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:59:18.049907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237fce2-56d0-4b71-9fd8-b84b0e4e5608 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:18.064577 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237fce2-56d0-4b71-9fd8-b84b0e4e5608 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:18.073132 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dbd58d-6d32-443f-ac83-6a862c036f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:18.074520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dbd58d-6d32-443f-ac83-6a862c036f6e admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:18.075227 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dbd58d-6d32-443f-ac83-6a862c036f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:18.075909 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dbd58d-6d32-443f-ac83-6a862c036f6e admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:18.077222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dbd58d-6d32-443f-ac83-6a862c036f6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:18.084636 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dbd58d-6d32-443f-ac83-6a862c036f6e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:18.147162 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1150/2318] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:59:18 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:59:18.148810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2ae0e-7ec7-4872-a472-7fad34827c7a admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:18.169592 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2ae0e-7ec7-4872-a472-7fad34827c7a admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:18.327616 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52271245-452d-42e3-bba8-5b29006e8cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:18.328334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52271245-452d-42e3-bba8-5b29006e8cb9 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:18.328721 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52271245-452d-42e3-bba8-5b29006e8cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:18.328951 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52271245-452d-42e3-bba8-5b29006e8cb9 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:18.329828 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52271245-452d-42e3-bba8-5b29006e8cb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:18.332041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52271245-452d-42e3-bba8-5b29006e8cb9 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:18.369086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1169/2319] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:59:18 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:59:18.373504 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dbd58d-6d32-443f-ac83-6a862c036f6e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:18.387120 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dbd58d-6d32-443f-ac83-6a862c036f6e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:18.392647 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d230033-4d6f-408b-bd71-335de89f684d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:18.393579 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d230033-4d6f-408b-bd71-335de89f684d admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:18.394026 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d230033-4d6f-408b-bd71-335de89f684d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:18.394476 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d230033-4d6f-408b-bd71-335de89f684d admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:18.395365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d230033-4d6f-408b-bd71-335de89f684d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:18.397577 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d230033-4d6f-408b-bd71-335de89f684d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:18.415377 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1151/2320] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 08:59:18.420445 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52271245-452d-42e3-bba8-5b29006e8cb9 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:18.439265 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52271245-452d-42e3-bba8-5b29006e8cb9 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:18.444591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98663145-6b37-499e-bad2-4be68a09b58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:18.445222 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98663145-6b37-499e-bad2-4be68a09b58b admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:18.445507 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98663145-6b37-499e-bad2-4be68a09b58b admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:18.445782 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98663145-6b37-499e-bad2-4be68a09b58b admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:18.446523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98663145-6b37-499e-bad2-4be68a09b58b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:18.448356 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98663145-6b37-499e-bad2-4be68a09b58b admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:18.467912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1170/2321] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:59:18.469890 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d230033-4d6f-408b-bd71-335de89f684d admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:18.488782 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d230033-4d6f-408b-bd71-335de89f684d admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:18.495276 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a34e23-4067-4e5c-9b7d-40e79f561067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:18.496478 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a34e23-4067-4e5c-9b7d-40e79f561067 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:18.496919 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a34e23-4067-4e5c-9b7d-40e79f561067 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:18.497472 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a34e23-4067-4e5c-9b7d-40e79f561067 admin admin] PATH_INFO: `/v3/projects/f7e75f903f7942dda7df96c0300baf46/users/71b7df583d74468a8a7ae47f0190c220/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:18.542763 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a34e23-4067-4e5c-9b7d-40e79f561067 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7e75f903f7942dda7df96c0300baf46, user_id=71b7df583d74468a8a7ae47f0190c220, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:18.546648 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a34e23-4067-4e5c-9b7d-40e79f561067 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:18.580586 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1152/2322] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:59:18 2022] PUT /identity/v3/projects/f7e75f903f7942dda7df96c0300baf46/users/71b7df583d74468a8a7ae47f0190c220/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:59:18.583589 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01a719-a1da-4bc9-8ebe-40996a0cb244 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:18.583991 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01a719-a1da-4bc9-8ebe-40996a0cb244 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:18.584294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01a719-a1da-4bc9-8ebe-40996a0cb244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:18.622440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-7d01a719-a1da-4bc9-8ebe-40996a0cb244 None None] MFA Rules not processed for user `71b7df583d74468a8a7ae47f0190c220`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:59:18.623651 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-7d01a719-a1da-4bc9-8ebe-40996a0cb244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:18.770084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1171/2323] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:59:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:59:18.783253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a34e23-4067-4e5c-9b7d-40e79f561067 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:18.786565 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-76a34e23-4067-4e5c-9b7d-40e79f561067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:18.886051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a34e23-4067-4e5c-9b7d-40e79f561067 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:18.894809 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e1963-043f-458d-bdfa-4d3ff27e91e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:18.901577 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e1963-043f-458d-bdfa-4d3ff27e91e5 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:18.901577 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e1963-043f-458d-bdfa-4d3ff27e91e5 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:18.901577 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e1963-043f-458d-bdfa-4d3ff27e91e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:18.908743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e1963-043f-458d-bdfa-4d3ff27e91e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:18.910394 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e1963-043f-458d-bdfa-4d3ff27e91e5 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:18.926967 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1153/2324] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:59:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:59:26.726020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459cda2-5bba-4208-b93c-60c4787f3ce4 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:26.726659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459cda2-5bba-4208-b93c-60c4787f3ce4 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:26.726659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459cda2-5bba-4208-b93c-60c4787f3ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:26.775392 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-9459cda2-5bba-4208-b93c-60c4787f3ce4 None None] MFA Rules not processed for user `df07cffd82c74621855c8b8d7bfa989d`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:59:26.777274 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-9459cda2-5bba-4208-b93c-60c4787f3ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:26.832965 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:59:26.909545 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1172/2325] 10.222.0.71 () {58 vars in 967 bytes} [Fri Aug 26 08:59:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:59:26.923283 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e1963-043f-458d-bdfa-4d3ff27e91e5 service nova] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:26.924855 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c30e1963-043f-458d-bdfa-4d3ff27e91e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:26.959035 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:59:27.054483 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e1963-043f-458d-bdfa-4d3ff27e91e5 service nova] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:27.062021 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c8c85-df2d-4cf5-99f2-446199338957 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1723e961375b45b7b3bd99ddeb69683f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:27.062648 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c8c85-df2d-4cf5-99f2-446199338957 service swift] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:27.062963 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c8c85-df2d-4cf5-99f2-446199338957 service swift] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:27.063240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c8c85-df2d-4cf5-99f2-446199338957 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:27.072481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c8c85-df2d-4cf5-99f2-446199338957 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:27.074407 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c8c85-df2d-4cf5-99f2-446199338957 service swift] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:27.083855 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1154/2326] 10.222.0.71 () {60 vars in 1384 bytes} [Fri Aug 26 08:59:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 08:59:27.841096 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459cda2-5bba-4208-b93c-60c4787f3ce4 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:27.999393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459cda2-5bba-4208-b93c-60c4787f3ce4 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:28.005070 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a73386-1979-410d-9385-24cf2da0d43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:28.005743 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a73386-1979-410d-9385-24cf2da0d43f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:28.006020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a73386-1979-410d-9385-24cf2da0d43f admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:28.006296 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a73386-1979-410d-9385-24cf2da0d43f admin admin] PATH_INFO: `/v3/users/71b7df583d74468a8a7ae47f0190c220` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:28.007526 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a73386-1979-410d-9385-24cf2da0d43f admin admin] RBAC: Authorizing `identity:delete_user(user_id=71b7df583d74468a8a7ae47f0190c220)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:28.009476 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a73386-1979-410d-9385-24cf2da0d43f admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:28.071677 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-06a73386-1979-410d-9385-24cf2da0d43f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71b7df583d74468a8a7ae47f0190c220'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:28.088744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-06a73386-1979-410d-9385-24cf2da0d43f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71b7df583d74468a8a7ae47f0190c220'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:28.100415 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-06a73386-1979-410d-9385-24cf2da0d43f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71b7df583d74468a8a7ae47f0190c220'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:28.109297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-06a73386-1979-410d-9385-24cf2da0d43f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71b7df583d74468a8a7ae47f0190c220'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:28.110079 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-06a73386-1979-410d-9385-24cf2da0d43f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71b7df583d74468a8a7ae47f0190c220'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:28.117315 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1173/2327] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:59:27 2022] DELETE /identity/v3/users/71b7df583d74468a8a7ae47f0190c220 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:59:28.122975 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27456a-21bf-4a17-bf22-c0d0a6ab9ba8 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:28.123901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27456a-21bf-4a17-bf22-c0d0a6ab9ba8 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:28.123901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27456a-21bf-4a17-bf22-c0d0a6ab9ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:28.177647 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-0b27456a-21bf-4a17-bf22-c0d0a6ab9ba8 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:59:28.178992 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0b27456a-21bf-4a17-bf22-c0d0a6ab9ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:28.252329 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1155/2328] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:59:28.262532 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a73386-1979-410d-9385-24cf2da0d43f admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:28.264105 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-06a73386-1979-410d-9385-24cf2da0d43f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:28.293709 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:59:28.337505 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a73386-1979-410d-9385-24cf2da0d43f admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:28.342337 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d409e53d-ccdf-4ad7-bc91-d2cb36405cb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:28.342990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409e53d-ccdf-4ad7-bc91-d2cb36405cb1 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:28.343274 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409e53d-ccdf-4ad7-bc91-d2cb36405cb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:28.343551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d409e53d-ccdf-4ad7-bc91-d2cb36405cb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:28.360759 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409e53d-ccdf-4ad7-bc91-d2cb36405cb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:28.364556 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d409e53d-ccdf-4ad7-bc91-d2cb36405cb1 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:28.385003 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1174/2329] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:59:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:59:29.277501 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27456a-21bf-4a17-bf22-c0d0a6ab9ba8 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:29.279827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0b27456a-21bf-4a17-bf22-c0d0a6ab9ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:29.302509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27456a-21bf-4a17-bf22-c0d0a6ab9ba8 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:29.324851 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07890d8-86a8-4e0d-bc21-ff4217fee2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:29.325616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07890d8-86a8-4e0d-bc21-ff4217fee2ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:29.325805 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07890d8-86a8-4e0d-bc21-ff4217fee2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:29.326067 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07890d8-86a8-4e0d-bc21-ff4217fee2ec admin admin] PATH_INFO: `/v3/projects/f7e75f903f7942dda7df96c0300baf46` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:29.328660 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07890d8-86a8-4e0d-bc21-ff4217fee2ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7e75f903f7942dda7df96c0300baf46)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:29.330995 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07890d8-86a8-4e0d-bc21-ff4217fee2ec admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:29.423035 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d07890d8-86a8-4e0d-bc21-ff4217fee2ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7e75f903f7942dda7df96c0300baf46', 'request_id': 'req-d07890d8-86a8-4e0d-bc21-ff4217fee2ec'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:29.435859 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d07890d8-86a8-4e0d-bc21-ff4217fee2ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7e75f903f7942dda7df96c0300baf46', 'request_id': 'req-d07890d8-86a8-4e0d-bc21-ff4217fee2ec'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:29.440909 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:59:29.455815 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d07890d8-86a8-4e0d-bc21-ff4217fee2ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7e75f903f7942dda7df96c0300baf46', 'request_id': 'req-d07890d8-86a8-4e0d-bc21-ff4217fee2ec'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:29.475635 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d07890d8-86a8-4e0d-bc21-ff4217fee2ec admin admin] The token cache is being invalidate because project f7e75f903f7942dda7df96c0300baf46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:59:29.475969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d07890d8-86a8-4e0d-bc21-ff4217fee2ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:29.476929 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d07890d8-86a8-4e0d-bc21-ff4217fee2ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:29.485354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1156/2330] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:59:29 2022] DELETE /identity/v3/projects/f7e75f903f7942dda7df96c0300baf46 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:59:29.490614 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37c318-537c-4253-af1a-d65cfcd2fe71 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:29.491210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37c318-537c-4253-af1a-d65cfcd2fe71 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:29.491475 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37c318-537c-4253-af1a-d65cfcd2fe71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:29.560324 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-4c37c318-537c-4253-af1a-d65cfcd2fe71 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:59:29.561718 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4c37c318-537c-4253-af1a-d65cfcd2fe71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:29.653750 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1175/2331] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:59:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:59:29.665474 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8a0c6-f8fa-46b8-b894-10f1c6f7c1a7 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:29.666747 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8a0c6-f8fa-46b8-b894-10f1c6f7c1a7 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:29.667055 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8a0c6-f8fa-46b8-b894-10f1c6f7c1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:29.708679 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-9bf8a0c6-f8fa-46b8-b894-10f1c6f7c1a7 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:59:29.710356 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9bf8a0c6-f8fa-46b8-b894-10f1c6f7c1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:29.724053 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37c318-537c-4253-af1a-d65cfcd2fe71 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:29.725057 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1157/2332] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:59:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:59:29.736170 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37c318-537c-4253-af1a-d65cfcd2fe71 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:29.741690 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1ee1a-c117-44fa-9ad3-b2463eca4193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:29.742545 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1ee1a-c117-44fa-9ad3-b2463eca4193 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:29.742948 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1ee1a-c117-44fa-9ad3-b2463eca4193 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:29.743358 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1ee1a-c117-44fa-9ad3-b2463eca4193 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:29.744278 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1ee1a-c117-44fa-9ad3-b2463eca4193 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:59:29.744923 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1ee1a-c117-44fa-9ad3-b2463eca4193 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:29.747917 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1ee1a-c117-44fa-9ad3-b2463eca4193 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:29.772264 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1176/2333] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:59:29.776651 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8a0c6-f8fa-46b8-b894-10f1c6f7c1a7 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:29.807148 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8a0c6-f8fa-46b8-b894-10f1c6f7c1a7 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:29.816564 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cb7c3-55eb-469a-b2c9-2cf11ad4e132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:29.817596 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cb7c3-55eb-469a-b2c9-2cf11ad4e132 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:29.818375 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cb7c3-55eb-469a-b2c9-2cf11ad4e132 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:29.819305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321cb7c3-55eb-469a-b2c9-2cf11ad4e132 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:29.820383 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321cb7c3-55eb-469a-b2c9-2cf11ad4e132 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:29.823007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321cb7c3-55eb-469a-b2c9-2cf11ad4e132 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:29.881229 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1158/2334] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:59:29 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:59:29.885202 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1ee1a-c117-44fa-9ad3-b2463eca4193 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:29.897834 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1ee1a-c117-44fa-9ad3-b2463eca4193 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:30.019412 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af78014-6010-4c9c-a843-a3dd7fc25944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:30.020889 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af78014-6010-4c9c-a843-a3dd7fc25944 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:30.021580 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af78014-6010-4c9c-a843-a3dd7fc25944 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:30.022528 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af78014-6010-4c9c-a843-a3dd7fc25944 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:30.024078 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af78014-6010-4c9c-a843-a3dd7fc25944 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:30.029615 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af78014-6010-4c9c-a843-a3dd7fc25944 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:30.082171 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1177/2335] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:59:29 2022] POST /identity/v3/users => generated 294 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:59:30.087729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cb7c3-55eb-469a-b2c9-2cf11ad4e132 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:30.113958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321cb7c3-55eb-469a-b2c9-2cf11ad4e132 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:30.128662 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c8e9e5-1217-4022-a74e-e2b49314d3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:30.129746 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8e9e5-1217-4022-a74e-e2b49314d3eb admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:30.130017 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8e9e5-1217-4022-a74e-e2b49314d3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:30.130557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8e9e5-1217-4022-a74e-e2b49314d3eb admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:30.131638 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c8e9e5-1217-4022-a74e-e2b49314d3eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:30.135175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c8e9e5-1217-4022-a74e-e2b49314d3eb admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:30.154670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1159/2336] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 08:59:30.160611 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af78014-6010-4c9c-a843-a3dd7fc25944 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:30.182252 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af78014-6010-4c9c-a843-a3dd7fc25944 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:30.194360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53d49b-a87a-4247-a72e-2e307fc8be25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:30.195851 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53d49b-a87a-4247-a72e-2e307fc8be25 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:30.196512 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53d49b-a87a-4247-a72e-2e307fc8be25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:30.196928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53d49b-a87a-4247-a72e-2e307fc8be25 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:30.197985 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e53d49b-a87a-4247-a72e-2e307fc8be25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:30.201941 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e53d49b-a87a-4247-a72e-2e307fc8be25 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:30.227426 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1178/2337] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:59:30.230448 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c8e9e5-1217-4022-a74e-e2b49314d3eb admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:30.243882 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c8e9e5-1217-4022-a74e-e2b49314d3eb admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:30.253078 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa9871-5130-4720-bae2-df8212f304f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:30.253986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa9871-5130-4720-bae2-df8212f304f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:30.254473 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa9871-5130-4720-bae2-df8212f304f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:30.254939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa9871-5130-4720-bae2-df8212f304f2 admin admin] PATH_INFO: `/v3/projects/99570407d4e644deb3c8af96ab003ad7/users/292cb7582d95480ca122473dd94bf107/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:30.277635 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa9871-5130-4720-bae2-df8212f304f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99570407d4e644deb3c8af96ab003ad7, user_id=292cb7582d95480ca122473dd94bf107, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:30.280488 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa9871-5130-4720-bae2-df8212f304f2 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:30.303638 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1160/2338] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:59:30 2022] PUT /identity/v3/projects/99570407d4e644deb3c8af96ab003ad7/users/292cb7582d95480ca122473dd94bf107/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:59:30.318660 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d7b17-eab8-491c-b42c-7d904c7c3ef9 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:30.321334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d7b17-eab8-491c-b42c-7d904c7c3ef9 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:30.321334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1d7b17-eab8-491c-b42c-7d904c7c3ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:30.341723 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:59:30.379302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-1a1d7b17-eab8-491c-b42c-7d904c7c3ef9 None None] MFA Rules not processed for user `292cb7582d95480ca122473dd94bf107`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:59:30.381831 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1a1d7b17-eab8-491c-b42c-7d904c7c3ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:30.514984 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1179/2339] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:59:30 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:59:30.521237 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa9871-5130-4720-bae2-df8212f304f2 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:30.646366 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa9871-5130-4720-bae2-df8212f304f2 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:30.658203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6567c028-7599-4c5a-b314-9227e9a3af58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:30.658203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6567c028-7599-4c5a-b314-9227e9a3af58 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:30.658203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6567c028-7599-4c5a-b314-9227e9a3af58 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:30.658203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6567c028-7599-4c5a-b314-9227e9a3af58 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:30.659124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6567c028-7599-4c5a-b314-9227e9a3af58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:30.660978 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6567c028-7599-4c5a-b314-9227e9a3af58 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:30.719773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1161/2340] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:59:30 2022] POST /identity/v3/projects => generated 387 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:59:30.725982 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d7b17-eab8-491c-b42c-7d904c7c3ef9 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:30.750118 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da845d6-8ebb-4b26-996c-acfaddd98e76 None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:30.750404 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da845d6-8ebb-4b26-996c-acfaddd98e76 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:30.750914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da845d6-8ebb-4b26-996c-acfaddd98e76 None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:30.752963 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1162/2341] 10.222.0.71 () {54 vars in 921 bytes} [Fri Aug 26 08:59:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:59:30.754951 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1d7b17-eab8-491c-b42c-7d904c7c3ef9 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:30.765027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b945637-c409-4eb1-b63d-1bc097b1d040 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:30.765305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b945637-c409-4eb1-b63d-1bc097b1d040 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:30.765697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b945637-c409-4eb1-b63d-1bc097b1d040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:30.821738 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-7b945637-c409-4eb1-b63d-1bc097b1d040 None None] MFA Rules not processed for user `f16d1768715a41648af3fe6afadd7477`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:59:30.823058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7b945637-c409-4eb1-b63d-1bc097b1d040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:30.867854 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b908b-b3fc-4fba-8aba-1d0a56d7aae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:30.869107 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b908b-b3fc-4fba-8aba-1d0a56d7aae4 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:30.869573 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b908b-b3fc-4fba-8aba-1d0a56d7aae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:30.870550 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b908b-b3fc-4fba-8aba-1d0a56d7aae4 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:30.871433 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b908b-b3fc-4fba-8aba-1d0a56d7aae4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:30.873988 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b908b-b3fc-4fba-8aba-1d0a56d7aae4 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:30.911744 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:59:30.913610 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1180/2342] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:59:30 2022] POST /identity/v3/users => generated 300 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:59:30.922330 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b908b-b3fc-4fba-8aba-1d0a56d7aae4 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:30.936723 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b908b-b3fc-4fba-8aba-1d0a56d7aae4 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:30.944735 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93dde09-4df8-4b58-b03b-24614b6be440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:30.945975 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93dde09-4df8-4b58-b03b-24614b6be440 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:30.946551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93dde09-4df8-4b58-b03b-24614b6be440 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:30.947370 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93dde09-4df8-4b58-b03b-24614b6be440 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:30.948211 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93dde09-4df8-4b58-b03b-24614b6be440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:30.953746 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93dde09-4df8-4b58-b03b-24614b6be440 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:30.985124 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1181/2343] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:59:30 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:59:30.990988 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93dde09-4df8-4b58-b03b-24614b6be440 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:30.993899 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1163/2344] 10.222.0.71 () {58 vars in 1018 bytes} [Fri Aug 26 08:59:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:59:31.007272 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93dde09-4df8-4b58-b03b-24614b6be440 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:31.008263 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b945637-c409-4eb1-b63d-1bc097b1d040 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:31.011964 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7b945637-c409-4eb1-b63d-1bc097b1d040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:31.013050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05a6e4-9f57-465f-905b-87ddeb032173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:31.014044 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05a6e4-9f57-465f-905b-87ddeb032173 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:31.014342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05a6e4-9f57-465f-905b-87ddeb032173 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:31.014620 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05a6e4-9f57-465f-905b-87ddeb032173 admin admin] PATH_INFO: `/v3/projects/a086be7fc8c045c19e73eee5d94c048d/users/b9081c16db6c4e3cba6dc6369af60640/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:31.035055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05a6e4-9f57-465f-905b-87ddeb032173 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a086be7fc8c045c19e73eee5d94c048d, user_id=b9081c16db6c4e3cba6dc6369af60640, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:31.039760 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05a6e4-9f57-465f-905b-87ddeb032173 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:31.073857 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1182/2345] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:59:30 2022] PUT /identity/v3/projects/a086be7fc8c045c19e73eee5d94c048d/users/b9081c16db6c4e3cba6dc6369af60640/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 08:59:31.078300 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05a6e4-9f57-465f-905b-87ddeb032173 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:31.127831 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b945637-c409-4eb1-b63d-1bc097b1d040 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:31.188792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:59:31.200222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3159dc-02c3-4021-a117-797db47f16b4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d178c717a3e49319545e70cb58cbbed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:31.201272 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3159dc-02c3-4021-a117-797db47f16b4 service glance] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:31.201678 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3159dc-02c3-4021-a117-797db47f16b4 service glance] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:31.201902 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3159dc-02c3-4021-a117-797db47f16b4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:31.204418 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05a6e4-9f57-465f-905b-87ddeb032173 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:31.210631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c221ae9-0860-4056-9886-8cc217de657d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:31.211309 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c221ae9-0860-4056-9886-8cc217de657d admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:31.211309 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c221ae9-0860-4056-9886-8cc217de657d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:31.211309 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c221ae9-0860-4056-9886-8cc217de657d admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:31.212154 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c221ae9-0860-4056-9886-8cc217de657d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:31.213778 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c221ae9-0860-4056-9886-8cc217de657d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:31.231336 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1183/2346] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:59:31.237130 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c221ae9-0860-4056-9886-8cc217de657d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:31.257360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c221ae9-0860-4056-9886-8cc217de657d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:31.266785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce93e3a-bf81-4631-8895-76fb4d0f3c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:31.267789 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce93e3a-bf81-4631-8895-76fb4d0f3c82 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:31.268142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce93e3a-bf81-4631-8895-76fb4d0f3c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:31.268521 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce93e3a-bf81-4631-8895-76fb4d0f3c82 admin admin] PATH_INFO: `/v3/projects/a086be7fc8c045c19e73eee5d94c048d/users/b9081c16db6c4e3cba6dc6369af60640/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:31.272382 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce93e3a-bf81-4631-8895-76fb4d0f3c82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a086be7fc8c045c19e73eee5d94c048d, user_id=b9081c16db6c4e3cba6dc6369af60640, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:31.276787 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce93e3a-bf81-4631-8895-76fb4d0f3c82 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:31.297577 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1184/2347] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:59:31 2022] PUT /identity/v3/projects/a086be7fc8c045c19e73eee5d94c048d/users/b9081c16db6c4e3cba6dc6369af60640/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:59:31.314672 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2723da-105f-4344-97c2-5528efe9a132 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:31.315233 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2723da-105f-4344-97c2-5528efe9a132 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:31.315700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2723da-105f-4344-97c2-5528efe9a132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:31.320379 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3159dc-02c3-4021-a117-797db47f16b4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:31.335977 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3159dc-02c3-4021-a117-797db47f16b4 service glance] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:31.348903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1164/2348] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:59:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 08:59:31.366636 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-2e2723da-105f-4344-97c2-5528efe9a132 None None] MFA Rules not processed for user `b9081c16db6c4e3cba6dc6369af60640`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:59:31.374218 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2e2723da-105f-4344-97c2-5528efe9a132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:31.493599 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111b111-8828-416a-bf05-86af394113be None None] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:31.494216 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111b111-8828-416a-bf05-86af394113be None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:31.494809 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111b111-8828-416a-bf05-86af394113be None None] PATH_INFO: `/` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:31.497097 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1165/2349] 10.222.0.71 () {54 vars in 921 bytes} [Fri Aug 26 08:59:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:59:31.498487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1185/2350] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:59:31 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:59:31.512207 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6d106-d9a4-49d0-9351-12e2ba643121 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:31.512490 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6d106-d9a4-49d0-9351-12e2ba643121 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:31.512765 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6d106-d9a4-49d0-9351-12e2ba643121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:31.517056 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2723da-105f-4344-97c2-5528efe9a132 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:31.519894 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2e2723da-105f-4344-97c2-5528efe9a132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:31.554635 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-8ac6d106-d9a4-49d0-9351-12e2ba643121 None None] MFA Rules not processed for user `4aa8938a504c4c4f8815478e520482d8`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:59:31.555982 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8ac6d106-d9a4-49d0-9351-12e2ba643121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:31.627765 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:59:31.681186 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2723da-105f-4344-97c2-5528efe9a132 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:31.682700 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1166/2351] 10.222.0.71 () {58 vars in 1018 bytes} [Fri Aug 26 08:59:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:59:31.695377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91801103-9a6e-4185-9fb2-ba3913ded36a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:31.697276 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91801103-9a6e-4185-9fb2-ba3913ded36a service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:31.699168 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6d106-d9a4-49d0-9351-12e2ba643121 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:31.701633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8ac6d106-d9a4-49d0-9351-12e2ba643121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:31.711921 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91801103-9a6e-4185-9fb2-ba3913ded36a service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:31.712465 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91801103-9a6e-4185-9fb2-ba3913ded36a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:31.725894 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6d106-d9a4-49d0-9351-12e2ba643121 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:31.730220 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b7001-17b0-452a-918d-7ac3d78d4445 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:31.735540 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b7001-17b0-452a-918d-7ac3d78d4445 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:31.735967 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b7001-17b0-452a-918d-7ac3d78d4445 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:31.736593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b7001-17b0-452a-918d-7ac3d78d4445 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:31.740989 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91801103-9a6e-4185-9fb2-ba3913ded36a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:31.745916 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91801103-9a6e-4185-9fb2-ba3913ded36a service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:31.751346 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b7001-17b0-452a-918d-7ac3d78d4445 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:31.754979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b7001-17b0-452a-918d-7ac3d78d4445 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:31.765833 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1186/2352] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:59:31 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:59:31.780241 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1167/2353] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:59:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:59:31.973925 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da5532-bd8b-4d39-926c-a881d4ceced9 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:31.974759 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da5532-bd8b-4d39-926c-a881d4ceced9 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:31.975423 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da5532-bd8b-4d39-926c-a881d4ceced9 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:31.979263 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1187/2354] 10.222.0.71 () {54 vars in 921 bytes} [Fri Aug 26 08:59:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 08:59:31.989932 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe727f64-4c88-4846-876b-31d0f8ca4ec1 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:31.990403 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe727f64-4c88-4846-876b-31d0f8ca4ec1 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:31.990873 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe727f64-4c88-4846-876b-31d0f8ca4ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:32.042079 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-fe727f64-4c88-4846-876b-31d0f8ca4ec1 None None] MFA Rules not processed for user `f16d1768715a41648af3fe6afadd7477`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:59:32.045050 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-fe727f64-4c88-4846-876b-31d0f8ca4ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:32.227513 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1168/2355] 10.222.0.71 () {58 vars in 1018 bytes} [Fri Aug 26 08:59:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:59:32.231365 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da5532-bd8b-4d39-926c-a881d4ceced9 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:32.276295 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da5532-bd8b-4d39-926c-a881d4ceced9 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:32.289623 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c5ab7d-b262-4389-8ced-d5cdbecd7445 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:32.289623 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5ab7d-b262-4389-8ced-d5cdbecd7445 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:32.289623 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5ab7d-b262-4389-8ced-d5cdbecd7445 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:32.289623 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5ab7d-b262-4389-8ced-d5cdbecd7445 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:32.309278 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c5ab7d-b262-4389-8ced-d5cdbecd7445 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:32.314819 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c5ab7d-b262-4389-8ced-d5cdbecd7445 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:32.340165 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1188/2356] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:59:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:59:32.881903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe727f64-4c88-4846-876b-31d0f8ca4ec1 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:32.979395 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe727f64-4c88-4846-876b-31d0f8ca4ec1 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:32.987228 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37f679-3b3f-49ed-8229-29e21da4f06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:32.988135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37f679-3b3f-49ed-8229-29e21da4f06d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:32.988422 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37f679-3b3f-49ed-8229-29e21da4f06d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:32.988814 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37f679-3b3f-49ed-8229-29e21da4f06d admin admin] PATH_INFO: `/v3/users/292cb7582d95480ca122473dd94bf107` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:32.990373 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c37f679-3b3f-49ed-8229-29e21da4f06d admin admin] RBAC: Authorizing `identity:delete_user(user_id=292cb7582d95480ca122473dd94bf107)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:32.994128 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c37f679-3b3f-49ed-8229-29e21da4f06d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:33.074302 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1c37f679-3b3f-49ed-8229-29e21da4f06d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '292cb7582d95480ca122473dd94bf107'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:33.084073 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1c37f679-3b3f-49ed-8229-29e21da4f06d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '292cb7582d95480ca122473dd94bf107'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:33.092816 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1c37f679-3b3f-49ed-8229-29e21da4f06d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '292cb7582d95480ca122473dd94bf107'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:33.094439 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1c37f679-3b3f-49ed-8229-29e21da4f06d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '292cb7582d95480ca122473dd94bf107'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:33.099383 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1c37f679-3b3f-49ed-8229-29e21da4f06d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '292cb7582d95480ca122473dd94bf107'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:33.105389 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:59:33.132050 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1169/2357] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:59:32 2022] DELETE /identity/v3/users/292cb7582d95480ca122473dd94bf107 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:59:33.136789 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b91969-fb4e-447f-a711-f66eec99c557 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:33.137310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b91969-fb4e-447f-a711-f66eec99c557 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:33.137310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b91969-fb4e-447f-a711-f66eec99c557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:33.163518 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-71b91969-fb4e-447f-a711-f66eec99c557 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:59:33.165355 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-71b91969-fb4e-447f-a711-f66eec99c557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:33.243973 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:59:33.265984 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1189/2358] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:59:33.277577 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37f679-3b3f-49ed-8229-29e21da4f06d admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:33.279508 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1c37f679-3b3f-49ed-8229-29e21da4f06d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:33.400977 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37f679-3b3f-49ed-8229-29e21da4f06d admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:33.407012 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee7d4d-af29-4425-a991-025f9f54a6d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:33.407841 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee7d4d-af29-4425-a991-025f9f54a6d7 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:33.408233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee7d4d-af29-4425-a991-025f9f54a6d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:33.410246 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee7d4d-af29-4425-a991-025f9f54a6d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:33.424945 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee7d4d-af29-4425-a991-025f9f54a6d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:33.426667 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee7d4d-af29-4425-a991-025f9f54a6d7 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:33.458625 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1170/2359] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:59:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:59:33.950278 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b91969-fb4e-447f-a711-f66eec99c557 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:33.952129 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-71b91969-fb4e-447f-a711-f66eec99c557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:33.965082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b91969-fb4e-447f-a711-f66eec99c557 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:33.970584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8ebbe-3e83-4e50-bd9b-09e64bf2859e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:33.971553 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8ebbe-3e83-4e50-bd9b-09e64bf2859e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:33.971988 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8ebbe-3e83-4e50-bd9b-09e64bf2859e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:33.972360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8ebbe-3e83-4e50-bd9b-09e64bf2859e admin admin] PATH_INFO: `/v3/projects/99570407d4e644deb3c8af96ab003ad7` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:33.974060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8ebbe-3e83-4e50-bd9b-09e64bf2859e admin admin] RBAC: Authorizing `identity:delete_project(project_id=99570407d4e644deb3c8af96ab003ad7)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:33.976244 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8ebbe-3e83-4e50-bd9b-09e64bf2859e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:34.059210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-36b8ebbe-3e83-4e50-bd9b-09e64bf2859e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99570407d4e644deb3c8af96ab003ad7', 'request_id': 'req-36b8ebbe-3e83-4e50-bd9b-09e64bf2859e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:34.069382 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-36b8ebbe-3e83-4e50-bd9b-09e64bf2859e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99570407d4e644deb3c8af96ab003ad7', 'request_id': 'req-36b8ebbe-3e83-4e50-bd9b-09e64bf2859e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:34.085720 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-36b8ebbe-3e83-4e50-bd9b-09e64bf2859e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99570407d4e644deb3c8af96ab003ad7', 'request_id': 'req-36b8ebbe-3e83-4e50-bd9b-09e64bf2859e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:34.100642 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-36b8ebbe-3e83-4e50-bd9b-09e64bf2859e admin admin] The token cache is being invalidate because project 99570407d4e644deb3c8af96ab003ad7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:59:34.101006 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-36b8ebbe-3e83-4e50-bd9b-09e64bf2859e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:34.101733 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-36b8ebbe-3e83-4e50-bd9b-09e64bf2859e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:34.103916 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1190/2360] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:59:33 2022] DELETE /identity/v3/projects/99570407d4e644deb3c8af96ab003ad7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:59:34.109188 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee7d4d-af29-4425-a991-025f9f54a6d7 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:34.122860 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee7d4d-af29-4425-a991-025f9f54a6d7 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:34.225841 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aaa280-230f-4cf5-b46d-dba166e89e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:34.226394 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aaa280-230f-4cf5-b46d-dba166e89e5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:34.226739 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aaa280-230f-4cf5-b46d-dba166e89e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:34.227079 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aaa280-230f-4cf5-b46d-dba166e89e5b admin admin] PATH_INFO: `/v3/users/b9081c16db6c4e3cba6dc6369af60640` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:34.228222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aaa280-230f-4cf5-b46d-dba166e89e5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9081c16db6c4e3cba6dc6369af60640)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:34.230213 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aaa280-230f-4cf5-b46d-dba166e89e5b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:34.306075 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-15aaa280-230f-4cf5-b46d-dba166e89e5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9081c16db6c4e3cba6dc6369af60640'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:34.319372 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-15aaa280-230f-4cf5-b46d-dba166e89e5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9081c16db6c4e3cba6dc6369af60640'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:34.331525 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-15aaa280-230f-4cf5-b46d-dba166e89e5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9081c16db6c4e3cba6dc6369af60640'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:34.332317 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-15aaa280-230f-4cf5-b46d-dba166e89e5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9081c16db6c4e3cba6dc6369af60640'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:34.332990 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-15aaa280-230f-4cf5-b46d-dba166e89e5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9081c16db6c4e3cba6dc6369af60640'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:34.366716 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1171/2361] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:59:34 2022] DELETE /identity/v3/users/b9081c16db6c4e3cba6dc6369af60640 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:59:35.170127 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8ebbe-3e83-4e50-bd9b-09e64bf2859e admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:35.173341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-36b8ebbe-3e83-4e50-bd9b-09e64bf2859e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:35.198628 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8ebbe-3e83-4e50-bd9b-09e64bf2859e admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:35.358838 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2245b00-d1b2-483a-8a39-10d2f5d449ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:35.360013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2245b00-d1b2-483a-8a39-10d2f5d449ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:35.361453 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2245b00-d1b2-483a-8a39-10d2f5d449ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:35.362008 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2245b00-d1b2-483a-8a39-10d2f5d449ee admin admin] PATH_INFO: `/v3/projects/a086be7fc8c045c19e73eee5d94c048d` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:35.364226 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2245b00-d1b2-483a-8a39-10d2f5d449ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=a086be7fc8c045c19e73eee5d94c048d)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:35.368331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2245b00-d1b2-483a-8a39-10d2f5d449ee admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:35.456700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b2245b00-d1b2-483a-8a39-10d2f5d449ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a086be7fc8c045c19e73eee5d94c048d', 'request_id': 'req-b2245b00-d1b2-483a-8a39-10d2f5d449ee'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:35.469084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b2245b00-d1b2-483a-8a39-10d2f5d449ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a086be7fc8c045c19e73eee5d94c048d', 'request_id': 'req-b2245b00-d1b2-483a-8a39-10d2f5d449ee'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:35.488682 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b2245b00-d1b2-483a-8a39-10d2f5d449ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a086be7fc8c045c19e73eee5d94c048d', 'request_id': 'req-b2245b00-d1b2-483a-8a39-10d2f5d449ee'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:35.527617 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b2245b00-d1b2-483a-8a39-10d2f5d449ee admin admin] The token cache is being invalidate because project a086be7fc8c045c19e73eee5d94c048d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:59:35.528621 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b2245b00-d1b2-483a-8a39-10d2f5d449ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:35.530381 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b2245b00-d1b2-483a-8a39-10d2f5d449ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:35.539631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1191/2362] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:59:35 2022] DELETE /identity/v3/projects/a086be7fc8c045c19e73eee5d94c048d => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:59:35.542871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1596e-0bf7-4384-b59c-61900511a19e None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:35.543835 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1596e-0bf7-4384-b59c-61900511a19e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:35.544260 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1596e-0bf7-4384-b59c-61900511a19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:35.564013 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:59:35.576158 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-def1596e-0bf7-4384-b59c-61900511a19e None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:59:35.577679 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-def1596e-0bf7-4384-b59c-61900511a19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:35.750773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1172/2363] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:59:35.753587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3938fb-8fed-4a56-978f-58f5edcc5c60 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:35.753936 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3938fb-8fed-4a56-978f-58f5edcc5c60 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:35.755374 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3938fb-8fed-4a56-978f-58f5edcc5c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:35.769273 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:59:35.806626 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-1f3938fb-8fed-4a56-978f-58f5edcc5c60 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:59:35.812672 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1f3938fb-8fed-4a56-978f-58f5edcc5c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:35.842798 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1192/2364] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:59:35.844711 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1596e-0bf7-4384-b59c-61900511a19e None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:35.867426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def1596e-0bf7-4384-b59c-61900511a19e None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:35.887109 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5815ef8-c85e-4a8c-b81c-a5f247a72c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:35.888313 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5815ef8-c85e-4a8c-b81c-a5f247a72c1e admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:35.888787 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5815ef8-c85e-4a8c-b81c-a5f247a72c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:35.889265 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5815ef8-c85e-4a8c-b81c-a5f247a72c1e admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:35.890437 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5815ef8-c85e-4a8c-b81c-a5f247a72c1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 08:59:35.891034 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5815ef8-c85e-4a8c-b81c-a5f247a72c1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:35.894184 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5815ef8-c85e-4a8c-b81c-a5f247a72c1e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:35.936817 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1173/2365] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 08:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 08:59:35.938962 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3938fb-8fed-4a56-978f-58f5edcc5c60 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:35.961087 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3938fb-8fed-4a56-978f-58f5edcc5c60 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:35.966492 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e39400-0956-4a22-8dbc-a9e9dbb3c31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:35.967205 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e39400-0956-4a22-8dbc-a9e9dbb3c31c admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:35.967482 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e39400-0956-4a22-8dbc-a9e9dbb3c31c admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:35.967762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e39400-0956-4a22-8dbc-a9e9dbb3c31c admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:35.968409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e39400-0956-4a22-8dbc-a9e9dbb3c31c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:35.970445 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e39400-0956-4a22-8dbc-a9e9dbb3c31c admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:36.030692 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1193/2366] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:59:35 2022] POST /identity/v3/projects => generated 373 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:59:36.032683 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5815ef8-c85e-4a8c-b81c-a5f247a72c1e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:36.057619 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5815ef8-c85e-4a8c-b81c-a5f247a72c1e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:36.153079 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b57940-b636-4cea-a1db-7016ed67438e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:36.153821 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b57940-b636-4cea-a1db-7016ed67438e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:36.154505 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b57940-b636-4cea-a1db-7016ed67438e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:36.202962 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-21b57940-b636-4cea-a1db-7016ed67438e None None] MFA Rules not processed for user `df07cffd82c74621855c8b8d7bfa989d`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:59:36.206734 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-21b57940-b636-4cea-a1db-7016ed67438e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:36.207795 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbfb4f-30d8-4f2d-b3f1-c0645efc6714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:36.208799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbfb4f-30d8-4f2d-b3f1-c0645efc6714 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:36.209583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbfb4f-30d8-4f2d-b3f1-c0645efc6714 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:36.210171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbfb4f-30d8-4f2d-b3f1-c0645efc6714 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:36.211436 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dbfb4f-30d8-4f2d-b3f1-c0645efc6714 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:36.214347 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dbfb4f-30d8-4f2d-b3f1-c0645efc6714 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:36.261747 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1174/2367] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:59:36 2022] POST /identity/v3/users => generated 287 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:59:36.266686 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbfb4f-30d8-4f2d-b3f1-c0645efc6714 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:36.293638 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbfb4f-30d8-4f2d-b3f1-c0645efc6714 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:36.323739 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9db067-1953-4cd0-b8fe-8f23c9c38f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:36.324687 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9db067-1953-4cd0-b8fe-8f23c9c38f13 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:36.327480 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9db067-1953-4cd0-b8fe-8f23c9c38f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:36.327973 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9db067-1953-4cd0-b8fe-8f23c9c38f13 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:36.330253 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9db067-1953-4cd0-b8fe-8f23c9c38f13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:36.337690 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9db067-1953-4cd0-b8fe-8f23c9c38f13 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:36.346429 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1194/2368] 10.222.0.71 () {58 vars in 1009 bytes} [Fri Aug 26 08:59:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 08:59:36.367149 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b57940-b636-4cea-a1db-7016ed67438e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:36.370724 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1175/2369] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:59:36.374488 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9db067-1953-4cd0-b8fe-8f23c9c38f13 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:36.382452 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-21b57940-b636-4cea-a1db-7016ed67438e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:36.420135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9db067-1953-4cd0-b8fe-8f23c9c38f13 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:36.442918 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2f3c9-9bde-4279-a40f-a75d027da680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:36.446193 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2f3c9-9bde-4279-a40f-a75d027da680 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:36.446193 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2f3c9-9bde-4279-a40f-a75d027da680 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:36.446193 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2f3c9-9bde-4279-a40f-a75d027da680 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:36.446697 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2f3c9-9bde-4279-a40f-a75d027da680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:36.453605 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a2f3c9-9bde-4279-a40f-a75d027da680 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:36.480788 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1176/2370] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 08:59:36.485847 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2f3c9-9bde-4279-a40f-a75d027da680 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:36.519846 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2f3c9-9bde-4279-a40f-a75d027da680 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:36.538335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e0e61-27e8-4d6b-9cf9-a59b4a85db6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:36.541279 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e0e61-27e8-4d6b-9cf9-a59b4a85db6e admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:36.541770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e0e61-27e8-4d6b-9cf9-a59b4a85db6e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:36.542240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e0e61-27e8-4d6b-9cf9-a59b4a85db6e admin admin] PATH_INFO: `/v3/projects/f8b63707f29c4aaa9f9c0293a53251c2/users/26675b5899084bb3be6f3e512a50d81e/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:36.597176 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193e0e61-27e8-4d6b-9cf9-a59b4a85db6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8b63707f29c4aaa9f9c0293a53251c2, user_id=26675b5899084bb3be6f3e512a50d81e, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:36.600364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193e0e61-27e8-4d6b-9cf9-a59b4a85db6e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:36.636263 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1177/2371] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:59:36 2022] PUT /identity/v3/projects/f8b63707f29c4aaa9f9c0293a53251c2/users/26675b5899084bb3be6f3e512a50d81e/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:59:36.636484 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b57940-b636-4cea-a1db-7016ed67438e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:36.639027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5636dea-836c-4159-a4a4-92c2cc64eebb None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:36.639317 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5636dea-836c-4159-a4a4-92c2cc64eebb None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:36.639651 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5636dea-836c-4159-a4a4-92c2cc64eebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:36.761433 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-e5636dea-836c-4159-a4a4-92c2cc64eebb None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:59:36.764178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e5636dea-836c-4159-a4a4-92c2cc64eebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:36.905003 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:59:36.923915 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a28000c-5046-450e-829d-c9c028a71013 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1723e961375b45b7b3bd99ddeb69683f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:36.924677 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28000c-5046-450e-829d-c9c028a71013 service swift] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:36.925215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28000c-5046-450e-829d-c9c028a71013 service swift] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:36.925750 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28000c-5046-450e-829d-c9c028a71013 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:36.984478 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:59:36.999066 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1178/2372] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:59:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:59:37.011026 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5636dea-836c-4159-a4a4-92c2cc64eebb None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:37.013270 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e5636dea-836c-4159-a4a4-92c2cc64eebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:37.148825 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a28000c-5046-450e-829d-c9c028a71013 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:37.154056 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a28000c-5046-450e-829d-c9c028a71013 service swift] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:37.216745 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1195/2373] 10.222.0.71 () {60 vars in 1384 bytes} [Fri Aug 26 08:59:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 849 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 08:59:37.220945 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5636dea-836c-4159-a4a4-92c2cc64eebb None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:37.234844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ae0cc-bbcc-4d8f-9b15-9728c3a4644d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:37.235571 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ae0cc-bbcc-4d8f-9b15-9728c3a4644d service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:37.235851 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ae0cc-bbcc-4d8f-9b15-9728c3a4644d service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:37.236126 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ae0cc-bbcc-4d8f-9b15-9728c3a4644d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:37.274671 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4ae0cc-bbcc-4d8f-9b15-9728c3a4644d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:37.277882 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4ae0cc-bbcc-4d8f-9b15-9728c3a4644d service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:37.325402 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1179/2374] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:59:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:59:38.206639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a28000c-5046-450e-829d-c9c028a71013 service swift] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:38.208964 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4a28000c-5046-450e-829d-c9c028a71013 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:38.228465 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a28000c-5046-450e-829d-c9c028a71013 service swift] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:38.236197 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915a31f-e954-4f04-a72e-fea52582c597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:38.237238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915a31f-e954-4f04-a72e-fea52582c597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:38.237981 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915a31f-e954-4f04-a72e-fea52582c597 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:38.238528 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915a31f-e954-4f04-a72e-fea52582c597 admin admin] PATH_INFO: `/v3/users/2cee3394dbdd4cb3a52005ee357d3be7` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:38.240373 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915a31f-e954-4f04-a72e-fea52582c597 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cee3394dbdd4cb3a52005ee357d3be7)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:38.242737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915a31f-e954-4f04-a72e-fea52582c597 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:38.308739 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c915a31f-e954-4f04-a72e-fea52582c597 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cee3394dbdd4cb3a52005ee357d3be7'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:38.323766 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c915a31f-e954-4f04-a72e-fea52582c597 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cee3394dbdd4cb3a52005ee357d3be7'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:38.334580 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c915a31f-e954-4f04-a72e-fea52582c597 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cee3394dbdd4cb3a52005ee357d3be7'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:38.343774 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c915a31f-e954-4f04-a72e-fea52582c597 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cee3394dbdd4cb3a52005ee357d3be7'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:38.344695 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c915a31f-e954-4f04-a72e-fea52582c597 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cee3394dbdd4cb3a52005ee357d3be7'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:38.351782 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1196/2375] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:59:38 2022] DELETE /identity/v3/users/2cee3394dbdd4cb3a52005ee357d3be7 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:59:38.541381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d887a8-97bb-4a8b-914c-e9432f62f452 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:38.542012 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d887a8-97bb-4a8b-914c-e9432f62f452 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:38.542575 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d887a8-97bb-4a8b-914c-e9432f62f452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:38.617980 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-58d887a8-97bb-4a8b-914c-e9432f62f452 None None] MFA Rules not processed for user `26675b5899084bb3be6f3e512a50d81e`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:59:38.622795 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-58d887a8-97bb-4a8b-914c-e9432f62f452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:38.682310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915a31f-e954-4f04-a72e-fea52582c597 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:38.684678 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c915a31f-e954-4f04-a72e-fea52582c597 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:38.748458 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1180/2376] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:59:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:59:38.752574 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d887a8-97bb-4a8b-914c-e9432f62f452 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:38.757193 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-58d887a8-97bb-4a8b-914c-e9432f62f452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:38.908852 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d887a8-97bb-4a8b-914c-e9432f62f452 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:38.917783 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16574ed3-aa6e-410e-9dc2-0efc83e4d496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:38.918908 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16574ed3-aa6e-410e-9dc2-0efc83e4d496 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:38.919393 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16574ed3-aa6e-410e-9dc2-0efc83e4d496 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:38.919858 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16574ed3-aa6e-410e-9dc2-0efc83e4d496 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:38.921224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16574ed3-aa6e-410e-9dc2-0efc83e4d496 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:38.923920 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915a31f-e954-4f04-a72e-fea52582c597 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:38.925066 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16574ed3-aa6e-410e-9dc2-0efc83e4d496 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:38.931831 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4591eea-2ecd-48cc-9952-0fd19a4ef774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:38.933718 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4591eea-2ecd-48cc-9952-0fd19a4ef774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:38.934386 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4591eea-2ecd-48cc-9952-0fd19a4ef774 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:38.934875 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4591eea-2ecd-48cc-9952-0fd19a4ef774 admin admin] PATH_INFO: `/v3/projects/3d18cb354a0f46faba6c00b89b4e4b1c` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:38.936913 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4591eea-2ecd-48cc-9952-0fd19a4ef774 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d18cb354a0f46faba6c00b89b4e4b1c)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:38.939778 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4591eea-2ecd-48cc-9952-0fd19a4ef774 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:38.969121 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:59:39.003686 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1181/2377] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 08:59:38 2022] POST /identity/v3/projects => generated 371 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 08:59:39.005695 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16574ed3-aa6e-410e-9dc2-0efc83e4d496 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:39.044024 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:59:39.057703 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b4591eea-2ecd-48cc-9952-0fd19a4ef774 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d18cb354a0f46faba6c00b89b4e4b1c', 'request_id': 'req-b4591eea-2ecd-48cc-9952-0fd19a4ef774'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:39.069687 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b4591eea-2ecd-48cc-9952-0fd19a4ef774 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d18cb354a0f46faba6c00b89b4e4b1c', 'request_id': 'req-b4591eea-2ecd-48cc-9952-0fd19a4ef774'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:39.090059 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b4591eea-2ecd-48cc-9952-0fd19a4ef774 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d18cb354a0f46faba6c00b89b4e4b1c', 'request_id': 'req-b4591eea-2ecd-48cc-9952-0fd19a4ef774'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:39.115041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b4591eea-2ecd-48cc-9952-0fd19a4ef774 admin admin] The token cache is being invalidate because project 3d18cb354a0f46faba6c00b89b4e4b1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:59:39.118581 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b4591eea-2ecd-48cc-9952-0fd19a4ef774 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:39.119981 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b4591eea-2ecd-48cc-9952-0fd19a4ef774 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:39.127740 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1197/2378] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:59:38 2022] DELETE /identity/v3/projects/3d18cb354a0f46faba6c00b89b4e4b1c => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:59:39.130214 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4591eea-2ecd-48cc-9952-0fd19a4ef774 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:39.133066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b4591eea-2ecd-48cc-9952-0fd19a4ef774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:39.197091 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16574ed3-aa6e-410e-9dc2-0efc83e4d496 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:39.281070 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4591eea-2ecd-48cc-9952-0fd19a4ef774 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:39.287205 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876c81c-eddc-4c3b-acd7-0f7bb5b5de77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:39.287933 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876c81c-eddc-4c3b-acd7-0f7bb5b5de77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:39.288367 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876c81c-eddc-4c3b-acd7-0f7bb5b5de77 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:39.288679 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876c81c-eddc-4c3b-acd7-0f7bb5b5de77 admin admin] PATH_INFO: `/v3/users/ef15b1a9f42a402d92f65fd4971f2e81` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:39.290356 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876c81c-eddc-4c3b-acd7-0f7bb5b5de77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef15b1a9f42a402d92f65fd4971f2e81)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:39.304078 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876c81c-eddc-4c3b-acd7-0f7bb5b5de77 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:39.322102 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a64ec-da80-4e59-a1a9-5bb9d77e2212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:39.323123 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a64ec-da80-4e59-a1a9-5bb9d77e2212 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:39.323409 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a64ec-da80-4e59-a1a9-5bb9d77e2212 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:39.323686 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a64ec-da80-4e59-a1a9-5bb9d77e2212 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:39.326565 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a64ec-da80-4e59-a1a9-5bb9d77e2212 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:39.330422 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03a64ec-da80-4e59-a1a9-5bb9d77e2212 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:39.396795 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1182/2379] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 08:59:39 2022] POST /identity/v3/users => generated 292 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 08:59:39.401734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a64ec-da80-4e59-a1a9-5bb9d77e2212 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:39.406708 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b03a64ec-da80-4e59-a1a9-5bb9d77e2212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:39.418054 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7876c81c-eddc-4c3b-acd7-0f7bb5b5de77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef15b1a9f42a402d92f65fd4971f2e81'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:39.441726 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7876c81c-eddc-4c3b-acd7-0f7bb5b5de77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef15b1a9f42a402d92f65fd4971f2e81'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:39.466395 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7876c81c-eddc-4c3b-acd7-0f7bb5b5de77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef15b1a9f42a402d92f65fd4971f2e81'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:39.477485 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a64ec-da80-4e59-a1a9-5bb9d77e2212 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:39.484955 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b67ad-49df-40d0-bfc5-ad22241111e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:39.485660 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b67ad-49df-40d0-bfc5-ad22241111e7 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:39.485956 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b67ad-49df-40d0-bfc5-ad22241111e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:39.486219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b67ad-49df-40d0-bfc5-ad22241111e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:39.487022 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b67ad-49df-40d0-bfc5-ad22241111e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:39.489043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b67ad-49df-40d0-bfc5-ad22241111e7 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:39.493848 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7876c81c-eddc-4c3b-acd7-0f7bb5b5de77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef15b1a9f42a402d92f65fd4971f2e81'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:39.495181 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7876c81c-eddc-4c3b-acd7-0f7bb5b5de77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef15b1a9f42a402d92f65fd4971f2e81'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:39.506173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1183/2380] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:59:39.511265 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b67ad-49df-40d0-bfc5-ad22241111e7 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:39.528914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-691b67ad-49df-40d0-bfc5-ad22241111e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:39.546321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1198/2381] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 08:59:39 2022] DELETE /identity/v3/users/ef15b1a9f42a402d92f65fd4971f2e81 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:59:39.583685 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b67ad-49df-40d0-bfc5-ad22241111e7 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:39.845404 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc361662-5b57-4041-979d-601cc359ebd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:39.846754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc361662-5b57-4041-979d-601cc359ebd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:39.847306 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc361662-5b57-4041-979d-601cc359ebd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:39.847784 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc361662-5b57-4041-979d-601cc359ebd1 admin admin] PATH_INFO: `/v3/projects/f6b67d4b9dfe46c79b2a1a5d2887ce3b/users/38426993616f489eac75f0d1c1d0e87e/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:39.938496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc361662-5b57-4041-979d-601cc359ebd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6b67d4b9dfe46c79b2a1a5d2887ce3b, user_id=38426993616f489eac75f0d1c1d0e87e, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:39.945251 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc361662-5b57-4041-979d-601cc359ebd1 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:40.006836 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1184/2382] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:59:39 2022] PUT /identity/v3/projects/f6b67d4b9dfe46c79b2a1a5d2887ce3b/users/38426993616f489eac75f0d1c1d0e87e/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:59:40.013546 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876c81c-eddc-4c3b-acd7-0f7bb5b5de77 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:40.074072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876c81c-eddc-4c3b-acd7-0f7bb5b5de77 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:40.207788 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:59:40.263131 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30dadd-3979-40f6-85b7-d6837e27dd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:40.264129 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30dadd-3979-40f6-85b7-d6837e27dd70 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:40.264610 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30dadd-3979-40f6-85b7-d6837e27dd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:40.265063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f30dadd-3979-40f6-85b7-d6837e27dd70 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:40.279029 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30dadd-3979-40f6-85b7-d6837e27dd70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:40.284016 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f30dadd-3979-40f6-85b7-d6837e27dd70 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:40.334846 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc361662-5b57-4041-979d-601cc359ebd1 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:40.335087 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1199/2383] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 08:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 08:59:40.369547 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:59:40.381432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc361662-5b57-4041-979d-601cc359ebd1 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:40.392240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b7d6a-a99e-43c6-beb6-d92cb91d494c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:40.393754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b7d6a-a99e-43c6-beb6-d92cb91d494c admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:40.397897 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b7d6a-a99e-43c6-beb6-d92cb91d494c admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:40.398593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b7d6a-a99e-43c6-beb6-d92cb91d494c admin admin] PATH_INFO: `/v3/projects/f6b67d4b9dfe46c79b2a1a5d2887ce3b/users/38426993616f489eac75f0d1c1d0e87e/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:40.407871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303b7d6a-a99e-43c6-beb6-d92cb91d494c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6b67d4b9dfe46c79b2a1a5d2887ce3b, user_id=38426993616f489eac75f0d1c1d0e87e, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:40.411814 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303b7d6a-a99e-43c6-beb6-d92cb91d494c admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:40.442968 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1185/2384] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 08:59:40 2022] PUT /identity/v3/projects/f6b67d4b9dfe46c79b2a1a5d2887ce3b/users/38426993616f489eac75f0d1c1d0e87e/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:59:40.552596 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30dadd-3979-40f6-85b7-d6837e27dd70 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:40.556131 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1f30dadd-3979-40f6-85b7-d6837e27dd70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:40.605786 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f30dadd-3979-40f6-85b7-d6837e27dd70 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:40.779640 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52a6a5-f849-4d76-8f6e-ca917c872306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:40.783686 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52a6a5-f849-4d76-8f6e-ca917c872306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:40.784019 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52a6a5-f849-4d76-8f6e-ca917c872306 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:40.784459 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52a6a5-f849-4d76-8f6e-ca917c872306 admin admin] PATH_INFO: `/v3/projects/5ffc549da2e2446889e2f0c0cc66e177` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:40.787430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c52a6a5-f849-4d76-8f6e-ca917c872306 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ffc549da2e2446889e2f0c0cc66e177)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:40.790353 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c52a6a5-f849-4d76-8f6e-ca917c872306 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:40.897644 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0c52a6a5-f849-4d76-8f6e-ca917c872306 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ffc549da2e2446889e2f0c0cc66e177', 'request_id': 'req-0c52a6a5-f849-4d76-8f6e-ca917c872306'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:40.909275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0c52a6a5-f849-4d76-8f6e-ca917c872306 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ffc549da2e2446889e2f0c0cc66e177', 'request_id': 'req-0c52a6a5-f849-4d76-8f6e-ca917c872306'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:40.935536 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0c52a6a5-f849-4d76-8f6e-ca917c872306 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ffc549da2e2446889e2f0c0cc66e177', 'request_id': 'req-0c52a6a5-f849-4d76-8f6e-ca917c872306'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:40.959527 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0c52a6a5-f849-4d76-8f6e-ca917c872306 admin admin] The token cache is being invalidate because project 5ffc549da2e2446889e2f0c0cc66e177 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 08:59:40.960169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0c52a6a5-f849-4d76-8f6e-ca917c872306 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:40.961070 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-0c52a6a5-f849-4d76-8f6e-ca917c872306 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 08:59:40.963585 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1200/2385] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 08:59:40 2022] DELETE /identity/v3/projects/5ffc549da2e2446889e2f0c0cc66e177 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 08:59:42.140421 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b7d6a-a99e-43c6-beb6-d92cb91d494c admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:42.142414 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-303b7d6a-a99e-43c6-beb6-d92cb91d494c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:42.421333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85d48b-0020-4432-95e6-a3605ce8dbad None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:42.422056 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85d48b-0020-4432-95e6-a3605ce8dbad None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:42.422174 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85d48b-0020-4432-95e6-a3605ce8dbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:42.484025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b7d6a-a99e-43c6-beb6-d92cb91d494c admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:42.564083 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-5d85d48b-0020-4432-95e6-a3605ce8dbad None None] MFA Rules not processed for user `38426993616f489eac75f0d1c1d0e87e`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:59:42.565359 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-5d85d48b-0020-4432-95e6-a3605ce8dbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:42.627836 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa6755-cc27-48c3-8699-0d4c5d2eb6da service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf5e2b4f5e440a7942adea3d51cec75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:42.631197 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa6755-cc27-48c3-8699-0d4c5d2eb6da service placement] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:42.631620 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa6755-cc27-48c3-8699-0d4c5d2eb6da service placement] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:42.631988 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa6755-cc27-48c3-8699-0d4c5d2eb6da service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:42.637480 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d0aa6755-cc27-48c3-8699-0d4c5d2eb6da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:42.662371 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aa6755-cc27-48c3-8699-0d4c5d2eb6da service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:42.666212 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aa6755-cc27-48c3-8699-0d4c5d2eb6da service placement] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:42.690806 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1186/2386] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:59:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:59:42.705449 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:59:42.723131 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1201/2387] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:59:42 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:59:42.739286 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa6755-cc27-48c3-8699-0d4c5d2eb6da service placement] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:42.742933 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d0aa6755-cc27-48c3-8699-0d4c5d2eb6da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:42.839792 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa6755-cc27-48c3-8699-0d4c5d2eb6da service placement] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:42.844547 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa552ef0-e030-4444-9905-18ce69863694 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:42.845442 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa552ef0-e030-4444-9905-18ce69863694 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:42.845824 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa552ef0-e030-4444-9905-18ce69863694 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:42.846320 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa552ef0-e030-4444-9905-18ce69863694 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:42.849330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-fa552ef0-e030-4444-9905-18ce69863694 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:42.879684 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:59:43.002331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa552ef0-e030-4444-9905-18ce69863694 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:43.005255 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa552ef0-e030-4444-9905-18ce69863694 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:43.023666 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1187/2388] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:59:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:59:44.788300 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85d48b-0020-4432-95e6-a3605ce8dbad None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:44.802402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85d48b-0020-4432-95e6-a3605ce8dbad None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:44.809494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf1e48-e81a-4840-a796-f51a581fcc65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:44.811346 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf1e48-e81a-4840-a796-f51a581fcc65 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:44.811346 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf1e48-e81a-4840-a796-f51a581fcc65 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:44.811346 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf1e48-e81a-4840-a796-f51a581fcc65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:44.823998 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf1e48-e81a-4840-a796-f51a581fcc65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:44.825797 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febf1e48-e81a-4840-a796-f51a581fcc65 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:44.837683 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1202/2389] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 08:59:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 08:59:53.012738 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=19 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:59:54.832900 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=8 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 08:59:55.836933 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea848ad5-79db-4cc2-84e7-a300f3aca3cf None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:55.838696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea848ad5-79db-4cc2-84e7-a300f3aca3cf None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:55.838696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea848ad5-79db-4cc2-84e7-a300f3aca3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:55.869436 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ea848ad5-79db-4cc2-84e7-a300f3aca3cf None None] MFA Rules not processed for user `406be06d6adf40bbb48d9baf12022988`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:59:55.871429 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ea848ad5-79db-4cc2-84e7-a300f3aca3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:55.959007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1188/2390] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 08:59:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:59:57.419258 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec3eb9-0736-4280-a1d8-2c0717a42772 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:57.422539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec3eb9-0736-4280-a1d8-2c0717a42772 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:57.423715 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec3eb9-0736-4280-a1d8-2c0717a42772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:57.453169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-4bec3eb9-0736-4280-a1d8-2c0717a42772 None None] MFA Rules not processed for user `df07cffd82c74621855c8b8d7bfa989d`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 08:59:57.459244 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-4bec3eb9-0736-4280-a1d8-2c0717a42772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:57.555214 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1203/2391] 10.222.0.71 () {58 vars in 967 bytes} [Fri Aug 26 08:59:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 08:59:57.572934 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea848ad5-79db-4cc2-84e7-a300f3aca3cf None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 08:59:57.574482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ea848ad5-79db-4cc2-84e7-a300f3aca3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 08:59:57.677283 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea848ad5-79db-4cc2-84e7-a300f3aca3cf None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 08:59:57.687175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09b2fb-68d7-41c6-8fef-004264865c78 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1723e961375b45b7b3bd99ddeb69683f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 08:59:57.688513 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09b2fb-68d7-41c6-8fef-004264865c78 service swift] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 08:59:57.689061 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09b2fb-68d7-41c6-8fef-004264865c78 service swift] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 08:59:57.689743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09b2fb-68d7-41c6-8fef-004264865c78 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 08:59:57.702227 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d09b2fb-68d7-41c6-8fef-004264865c78 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 08:59:57.704581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d09b2fb-68d7-41c6-8fef-004264865c78 service swift] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 08:59:57.715897 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1189/2392] 10.222.0.71 () {60 vars in 1384 bytes} [Fri Aug 26 08:59:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 09:00:00.428727 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0468edf-8ec9-49c7-a583-d54430303463 None None] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:00.430255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0468edf-8ec9-49c7-a583-d54430303463 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:00.430918 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0468edf-8ec9-49c7-a583-d54430303463 None None] PATH_INFO: `/` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:00.433407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1204/2393] 10.222.0.71 () {54 vars in 921 bytes} [Fri Aug 26 09:00:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 09:00:00.455771 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d91965-747d-43f2-89d2-5b2548c0955d None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:00.455771 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d91965-747d-43f2-89d2-5b2548c0955d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:00.458026 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d91965-747d-43f2-89d2-5b2548c0955d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:00.503198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0468edf-8ec9-49c7-a583-d54430303463 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:00.509039 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f0468edf-8ec9-49c7-a583-d54430303463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:00.549501 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-27d91965-747d-43f2-89d2-5b2548c0955d None None] MFA Rules not processed for user `0cf5e2b4f5e440a7942adea3d51cec75`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:00.553241 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-27d91965-747d-43f2-89d2-5b2548c0955d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:00.588359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1190/2394] 10.222.0.71 () {58 vars in 1018 bytes} [Fri Aug 26 09:00:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:00.604277 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d91965-747d-43f2-89d2-5b2548c0955d None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:00.606488 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-27d91965-747d-43f2-89d2-5b2548c0955d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:00.634663 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d91965-747d-43f2-89d2-5b2548c0955d None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:00.654325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a4560-f150-4aa4-bd4b-7fca40a07230 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf5e2b4f5e440a7942adea3d51cec75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:00.655309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a4560-f150-4aa4-bd4b-7fca40a07230 service placement] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:00.656027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a4560-f150-4aa4-bd4b-7fca40a07230 service placement] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:00.656538 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a4560-f150-4aa4-bd4b-7fca40a07230 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:00.680058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63a4560-f150-4aa4-bd4b-7fca40a07230 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:00.682345 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63a4560-f150-4aa4-bd4b-7fca40a07230 service placement] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:00.701297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1191/2395] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:00:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 09:00:00.704206 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0468edf-8ec9-49c7-a583-d54430303463 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:00.720821 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a897d586-f1c6-47b3-b73d-a272de26f4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:00.722182 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a897d586-f1c6-47b3-b73d-a272de26f4b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:00.722459 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a897d586-f1c6-47b3-b73d-a272de26f4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:00.723976 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a897d586-f1c6-47b3-b73d-a272de26f4b2 admin admin] PATH_INFO: `/v3/users/26675b5899084bb3be6f3e512a50d81e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:00.725695 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a897d586-f1c6-47b3-b73d-a272de26f4b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26675b5899084bb3be6f3e512a50d81e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:00.729683 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a897d586-f1c6-47b3-b73d-a272de26f4b2 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:00.799818 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a897d586-f1c6-47b3-b73d-a272de26f4b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26675b5899084bb3be6f3e512a50d81e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:00.825432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a897d586-f1c6-47b3-b73d-a272de26f4b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26675b5899084bb3be6f3e512a50d81e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:00.839616 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a897d586-f1c6-47b3-b73d-a272de26f4b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26675b5899084bb3be6f3e512a50d81e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:00.855360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a897d586-f1c6-47b3-b73d-a272de26f4b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26675b5899084bb3be6f3e512a50d81e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:00.856440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-a897d586-f1c6-47b3-b73d-a272de26f4b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26675b5899084bb3be6f3e512a50d81e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:00.870970 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1205/2396] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:00:00 2022] DELETE /identity/v3/users/26675b5899084bb3be6f3e512a50d81e => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:01.292961 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a4560-f150-4aa4-bd4b-7fca40a07230 service placement] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:01.328856 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a4560-f150-4aa4-bd4b-7fca40a07230 service placement] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:01.447853 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f7ff6-23bb-4f53-977c-47b59c39f65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:01.448808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f7ff6-23bb-4f53-977c-47b59c39f65e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:01.449739 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f7ff6-23bb-4f53-977c-47b59c39f65e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:01.450098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f7ff6-23bb-4f53-977c-47b59c39f65e admin admin] PATH_INFO: `/v3/projects/f8b63707f29c4aaa9f9c0293a53251c2` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:01.451491 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f7ff6-23bb-4f53-977c-47b59c39f65e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8b63707f29c4aaa9f9c0293a53251c2)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:01.453811 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f7ff6-23bb-4f53-977c-47b59c39f65e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:01.458248 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:01.549418 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-334f7ff6-23bb-4f53-977c-47b59c39f65e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8b63707f29c4aaa9f9c0293a53251c2', 'request_id': 'req-334f7ff6-23bb-4f53-977c-47b59c39f65e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:01.564544 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-334f7ff6-23bb-4f53-977c-47b59c39f65e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8b63707f29c4aaa9f9c0293a53251c2', 'request_id': 'req-334f7ff6-23bb-4f53-977c-47b59c39f65e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:01.585417 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-334f7ff6-23bb-4f53-977c-47b59c39f65e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8b63707f29c4aaa9f9c0293a53251c2', 'request_id': 'req-334f7ff6-23bb-4f53-977c-47b59c39f65e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:01.629964 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-334f7ff6-23bb-4f53-977c-47b59c39f65e admin admin] The token cache is being invalidate because project f8b63707f29c4aaa9f9c0293a53251c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:00:01.630245 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-334f7ff6-23bb-4f53-977c-47b59c39f65e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:01.631240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-334f7ff6-23bb-4f53-977c-47b59c39f65e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:01.633766 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1192/2397] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:00:01 2022] DELETE /identity/v3/projects/f8b63707f29c4aaa9f9c0293a53251c2 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:01.638497 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a897d586-f1c6-47b3-b73d-a272de26f4b2 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:01.640312 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a897d586-f1c6-47b3-b73d-a272de26f4b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:01.671902 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a897d586-f1c6-47b3-b73d-a272de26f4b2 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:01.710185 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:01.824511 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2503197f-1efa-470b-b0bc-ae8d70f607ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:01.825241 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503197f-1efa-470b-b0bc-ae8d70f607ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:01.825526 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503197f-1efa-470b-b0bc-ae8d70f607ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:01.825912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503197f-1efa-470b-b0bc-ae8d70f607ba admin admin] PATH_INFO: `/v3/users/38426993616f489eac75f0d1c1d0e87e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:01.828341 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2503197f-1efa-470b-b0bc-ae8d70f607ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=38426993616f489eac75f0d1c1d0e87e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:01.836405 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2503197f-1efa-470b-b0bc-ae8d70f607ba admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:01.972852 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2503197f-1efa-470b-b0bc-ae8d70f607ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38426993616f489eac75f0d1c1d0e87e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:02.028358 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2503197f-1efa-470b-b0bc-ae8d70f607ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38426993616f489eac75f0d1c1d0e87e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:02.045699 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2503197f-1efa-470b-b0bc-ae8d70f607ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38426993616f489eac75f0d1c1d0e87e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:02.062721 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2503197f-1efa-470b-b0bc-ae8d70f607ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38426993616f489eac75f0d1c1d0e87e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:02.063849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2503197f-1efa-470b-b0bc-ae8d70f607ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38426993616f489eac75f0d1c1d0e87e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:02.094714 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1206/2398] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:00:01 2022] DELETE /identity/v3/users/38426993616f489eac75f0d1c1d0e87e => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:02.439446 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f7ff6-23bb-4f53-977c-47b59c39f65e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:02.441768 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-334f7ff6-23bb-4f53-977c-47b59c39f65e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:02.467230 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f7ff6-23bb-4f53-977c-47b59c39f65e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:02.587839 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282eaa9-78a0-4115-a301-1f102583f5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:02.589835 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282eaa9-78a0-4115-a301-1f102583f5ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:02.590812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282eaa9-78a0-4115-a301-1f102583f5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:02.591838 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282eaa9-78a0-4115-a301-1f102583f5ec admin admin] PATH_INFO: `/v3/projects/f6b67d4b9dfe46c79b2a1a5d2887ce3b` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:02.609437 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282eaa9-78a0-4115-a301-1f102583f5ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6b67d4b9dfe46c79b2a1a5d2887ce3b)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:02.616774 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282eaa9-78a0-4115-a301-1f102583f5ec admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:02.752039 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a282eaa9-78a0-4115-a301-1f102583f5ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6b67d4b9dfe46c79b2a1a5d2887ce3b', 'request_id': 'req-a282eaa9-78a0-4115-a301-1f102583f5ec'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:02.767075 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a282eaa9-78a0-4115-a301-1f102583f5ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6b67d4b9dfe46c79b2a1a5d2887ce3b', 'request_id': 'req-a282eaa9-78a0-4115-a301-1f102583f5ec'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:02.776546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a282eaa9-78a0-4115-a301-1f102583f5ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6b67d4b9dfe46c79b2a1a5d2887ce3b', 'request_id': 'req-a282eaa9-78a0-4115-a301-1f102583f5ec'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:02.791333 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a282eaa9-78a0-4115-a301-1f102583f5ec admin admin] The token cache is being invalidate because project f6b67d4b9dfe46c79b2a1a5d2887ce3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:00:02.791906 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a282eaa9-78a0-4115-a301-1f102583f5ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:02.792837 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-a282eaa9-78a0-4115-a301-1f102583f5ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:02.799269 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1193/2399] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:00:02 2022] DELETE /identity/v3/projects/f6b67d4b9dfe46c79b2a1a5d2887ce3b => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:02.801970 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708e426-1020-458f-a569-b84c6e09dece None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:02.802602 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708e426-1020-458f-a569-b84c6e09dece None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:02.804466 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708e426-1020-458f-a569-b84c6e09dece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:02.847804 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e708e426-1020-458f-a569-b84c6e09dece None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:02.850153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e708e426-1020-458f-a569-b84c6e09dece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:02.947361 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1207/2400] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:02.958424 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3085d43-2e11-4526-93cc-599a2bfd2abf None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:02.958744 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3085d43-2e11-4526-93cc-599a2bfd2abf None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:02.959022 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3085d43-2e11-4526-93cc-599a2bfd2abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:02.987058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c3085d43-2e11-4526-93cc-599a2bfd2abf None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:02.988397 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c3085d43-2e11-4526-93cc-599a2bfd2abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:03.000227 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1194/2401] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 09:00:03.007463 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708e426-1020-458f-a569-b84c6e09dece None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:03.031127 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708e426-1020-458f-a569-b84c6e09dece None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:03.039844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d7375-2da4-4d10-b3cc-3e4383ec1dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:03.040733 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d7375-2da4-4d10-b3cc-3e4383ec1dfb admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:03.043329 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d7375-2da4-4d10-b3cc-3e4383ec1dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:03.043485 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d7375-2da4-4d10-b3cc-3e4383ec1dfb admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:03.044182 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d7375-2da4-4d10-b3cc-3e4383ec1dfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 09:00:03.044830 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d7375-2da4-4d10-b3cc-3e4383ec1dfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:03.046808 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026d7375-2da4-4d10-b3cc-3e4383ec1dfb admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:03.065416 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1208/2402] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 09:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 09:00:03.071071 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3085d43-2e11-4526-93cc-599a2bfd2abf None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:03.085284 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3085d43-2e11-4526-93cc-599a2bfd2abf None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:03.099746 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2cada5-0c03-464d-aee9-14014b83e1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:03.100436 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2cada5-0c03-464d-aee9-14014b83e1f4 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:03.100719 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2cada5-0c03-464d-aee9-14014b83e1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:03.100988 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2cada5-0c03-464d-aee9-14014b83e1f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:03.101669 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2cada5-0c03-464d-aee9-14014b83e1f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:03.108816 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2cada5-0c03-464d-aee9-14014b83e1f4 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:03.133124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:03.168275 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1195/2403] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:00:03 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:00:03.173434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d7375-2da4-4d10-b3cc-3e4383ec1dfb admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:03.189879 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d7375-2da4-4d10-b3cc-3e4383ec1dfb admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:03.259584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:03.307587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cfddc8-0c35-4c34-8ff0-b09094ff2f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:03.308627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfddc8-0c35-4c34-8ff0-b09094ff2f48 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:03.309222 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfddc8-0c35-4c34-8ff0-b09094ff2f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:03.309570 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfddc8-0c35-4c34-8ff0-b09094ff2f48 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:03.310434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cfddc8-0c35-4c34-8ff0-b09094ff2f48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:03.312611 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cfddc8-0c35-4c34-8ff0-b09094ff2f48 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:03.355119 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1209/2404] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:00:03 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:00:03.357220 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2cada5-0c03-464d-aee9-14014b83e1f4 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:03.389905 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2cada5-0c03-464d-aee9-14014b83e1f4 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:03.395886 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b8b764-16d5-4f88-98cb-d412b00b1402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:03.396531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8b764-16d5-4f88-98cb-d412b00b1402 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:03.396808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8b764-16d5-4f88-98cb-d412b00b1402 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:03.397153 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8b764-16d5-4f88-98cb-d412b00b1402 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:03.397908 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b8b764-16d5-4f88-98cb-d412b00b1402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:03.400077 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b8b764-16d5-4f88-98cb-d412b00b1402 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:03.414927 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1196/2405] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:00:03.418600 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cfddc8-0c35-4c34-8ff0-b09094ff2f48 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:03.437267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cfddc8-0c35-4c34-8ff0-b09094ff2f48 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:03.443562 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2077a0-cd11-40a9-ae03-95065f7066c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:03.444611 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2077a0-cd11-40a9-ae03-95065f7066c4 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:03.445089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2077a0-cd11-40a9-ae03-95065f7066c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:03.445982 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2077a0-cd11-40a9-ae03-95065f7066c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:03.454284 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2077a0-cd11-40a9-ae03-95065f7066c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:03.457660 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2077a0-cd11-40a9-ae03-95065f7066c4 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:03.482420 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1210/2406] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:00:03.483953 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b8b764-16d5-4f88-98cb-d412b00b1402 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:03.495105 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b8b764-16d5-4f88-98cb-d412b00b1402 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:03.500428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51026ff-05b6-4060-9198-71f7f4f9c472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:03.501496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51026ff-05b6-4060-9198-71f7f4f9c472 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:03.501734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51026ff-05b6-4060-9198-71f7f4f9c472 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:03.502067 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51026ff-05b6-4060-9198-71f7f4f9c472 admin admin] PATH_INFO: `/v3/projects/d8c9a9bf27664c4aa34c5637fa7504bb/users/1f3b1bddaf464731a4153c09516fd02e/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:03.524367 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51026ff-05b6-4060-9198-71f7f4f9c472 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8c9a9bf27664c4aa34c5637fa7504bb, user_id=1f3b1bddaf464731a4153c09516fd02e, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:03.527123 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51026ff-05b6-4060-9198-71f7f4f9c472 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:03.547048 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1197/2407] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:00:03 2022] PUT /identity/v3/projects/d8c9a9bf27664c4aa34c5637fa7504bb/users/1f3b1bddaf464731a4153c09516fd02e/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:03.559459 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b503a5e-9b2b-4a20-b9b9-2cbada734ba7 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:03.559759 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b503a5e-9b2b-4a20-b9b9-2cbada734ba7 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:03.559997 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b503a5e-9b2b-4a20-b9b9-2cbada734ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:03.594416 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-8b503a5e-9b2b-4a20-b9b9-2cbada734ba7 None None] MFA Rules not processed for user `1f3b1bddaf464731a4153c09516fd02e`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:03.595776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8b503a5e-9b2b-4a20-b9b9-2cbada734ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:03.701992 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1211/2408] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:03.719051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51026ff-05b6-4060-9198-71f7f4f9c472 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:03.723231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e51026ff-05b6-4060-9198-71f7f4f9c472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:03.842770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51026ff-05b6-4060-9198-71f7f4f9c472 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:03.847234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c43e25-a287-4ae1-a78b-7698353aa789 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:03.847975 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c43e25-a287-4ae1-a78b-7698353aa789 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:03.848293 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c43e25-a287-4ae1-a78b-7698353aa789 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:03.848577 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c43e25-a287-4ae1-a78b-7698353aa789 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:03.859622 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c43e25-a287-4ae1-a78b-7698353aa789 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:03.861422 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c43e25-a287-4ae1-a78b-7698353aa789 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:03.872640 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1198/2409] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:00:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:00:03.895096 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b503a5e-9b2b-4a20-b9b9-2cbada734ba7 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:03.974677 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b503a5e-9b2b-4a20-b9b9-2cbada734ba7 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:03.981416 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49899719-d2ad-490c-91f3-0a5aac2f4961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:03.982229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49899719-d2ad-490c-91f3-0a5aac2f4961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:03.982837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49899719-d2ad-490c-91f3-0a5aac2f4961 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:03.982837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49899719-d2ad-490c-91f3-0a5aac2f4961 admin admin] PATH_INFO: `/v3/users/1f3b1bddaf464731a4153c09516fd02e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:03.983950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49899719-d2ad-490c-91f3-0a5aac2f4961 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f3b1bddaf464731a4153c09516fd02e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:03.986039 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49899719-d2ad-490c-91f3-0a5aac2f4961 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:04.045461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:04.053004 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-49899719-d2ad-490c-91f3-0a5aac2f4961 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f3b1bddaf464731a4153c09516fd02e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:04.073492 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-49899719-d2ad-490c-91f3-0a5aac2f4961 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f3b1bddaf464731a4153c09516fd02e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:04.083998 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-49899719-d2ad-490c-91f3-0a5aac2f4961 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f3b1bddaf464731a4153c09516fd02e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:04.099455 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-49899719-d2ad-490c-91f3-0a5aac2f4961 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f3b1bddaf464731a4153c09516fd02e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:04.100315 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-49899719-d2ad-490c-91f3-0a5aac2f4961 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f3b1bddaf464731a4153c09516fd02e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:04.112929 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1212/2410] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:00:03 2022] DELETE /identity/v3/users/1f3b1bddaf464731a4153c09516fd02e => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:04.130327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ba2b6-bdc8-43e3-af38-b04eeb53d20a None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:04.130643 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ba2b6-bdc8-43e3-af38-b04eeb53d20a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:04.131029 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ba2b6-bdc8-43e3-af38-b04eeb53d20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:04.153947 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-474ba2b6-bdc8-43e3-af38-b04eeb53d20a None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:04.155393 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-474ba2b6-bdc8-43e3-af38-b04eeb53d20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:04.225750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1199/2411] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:04.236474 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49899719-d2ad-490c-91f3-0a5aac2f4961 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:04.238324 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-49899719-d2ad-490c-91f3-0a5aac2f4961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:04.336273 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49899719-d2ad-490c-91f3-0a5aac2f4961 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:04.340407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3201b1a4-7e8e-4593-accc-fbe15dbdc673 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:04.341039 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201b1a4-7e8e-4593-accc-fbe15dbdc673 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:04.341317 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201b1a4-7e8e-4593-accc-fbe15dbdc673 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:04.341585 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201b1a4-7e8e-4593-accc-fbe15dbdc673 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:04.352405 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3201b1a4-7e8e-4593-accc-fbe15dbdc673 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:04.353883 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3201b1a4-7e8e-4593-accc-fbe15dbdc673 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:04.370402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1213/2412] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:00:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:00:04.895613 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ba2b6-bdc8-43e3-af38-b04eeb53d20a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:04.899939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-474ba2b6-bdc8-43e3-af38-b04eeb53d20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:04.929586 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ba2b6-bdc8-43e3-af38-b04eeb53d20a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:04.941604 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7877f0d-5da2-42d6-87aa-ed54f83de8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:04.943387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7877f0d-5da2-42d6-87aa-ed54f83de8ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:04.944213 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7877f0d-5da2-42d6-87aa-ed54f83de8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:04.944989 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7877f0d-5da2-42d6-87aa-ed54f83de8ea admin admin] PATH_INFO: `/v3/projects/d8c9a9bf27664c4aa34c5637fa7504bb` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:04.947977 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7877f0d-5da2-42d6-87aa-ed54f83de8ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8c9a9bf27664c4aa34c5637fa7504bb)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:04.951453 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7877f0d-5da2-42d6-87aa-ed54f83de8ea admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:05.012204 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:INSERT=2,keystone:SELECT=95,keystone:DELETE=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:05.023804 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e7877f0d-5da2-42d6-87aa-ed54f83de8ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8c9a9bf27664c4aa34c5637fa7504bb', 'request_id': 'req-e7877f0d-5da2-42d6-87aa-ed54f83de8ea'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:05.037638 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e7877f0d-5da2-42d6-87aa-ed54f83de8ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8c9a9bf27664c4aa34c5637fa7504bb', 'request_id': 'req-e7877f0d-5da2-42d6-87aa-ed54f83de8ea'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:05.048262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e7877f0d-5da2-42d6-87aa-ed54f83de8ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8c9a9bf27664c4aa34c5637fa7504bb', 'request_id': 'req-e7877f0d-5da2-42d6-87aa-ed54f83de8ea'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:05.066572 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e7877f0d-5da2-42d6-87aa-ed54f83de8ea admin admin] The token cache is being invalidate because project d8c9a9bf27664c4aa34c5637fa7504bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:00:05.067027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e7877f0d-5da2-42d6-87aa-ed54f83de8ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:05.068647 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e7877f0d-5da2-42d6-87aa-ed54f83de8ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:05.070727 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1200/2413] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:00:04 2022] DELETE /identity/v3/projects/d8c9a9bf27664c4aa34c5637fa7504bb => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:05.077002 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740aec92-b181-48e2-91d5-564555f0e1af None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:05.078148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740aec92-b181-48e2-91d5-564555f0e1af None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:05.078148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740aec92-b181-48e2-91d5-564555f0e1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:05.106981 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-740aec92-b181-48e2-91d5-564555f0e1af None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:05.108435 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-740aec92-b181-48e2-91d5-564555f0e1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:05.196863 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1214/2414] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:05.207031 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789d53d-4137-49e7-a48d-22e71f252621 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:05.207536 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789d53d-4137-49e7-a48d-22e71f252621 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:05.207966 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789d53d-4137-49e7-a48d-22e71f252621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:05.242656 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-7789d53d-4137-49e7-a48d-22e71f252621 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:05.244593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7789d53d-4137-49e7-a48d-22e71f252621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:05.255896 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1201/2415] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:05.260539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740aec92-b181-48e2-91d5-564555f0e1af None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:05.274633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740aec92-b181-48e2-91d5-564555f0e1af None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:05.281202 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d54653-d427-48fb-bd89-c2963f9d2cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:05.282240 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d54653-d427-48fb-bd89-c2963f9d2cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:05.282714 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d54653-d427-48fb-bd89-c2963f9d2cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:05.283226 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d54653-d427-48fb-bd89-c2963f9d2cf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:05.284676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d54653-d427-48fb-bd89-c2963f9d2cf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 09:00:05.285347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d54653-d427-48fb-bd89-c2963f9d2cf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:05.288573 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d54653-d427-48fb-bd89-c2963f9d2cf6 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:05.309684 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1215/2416] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 09:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 09:00:05.311216 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789d53d-4137-49e7-a48d-22e71f252621 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:05.323438 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789d53d-4137-49e7-a48d-22e71f252621 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:05.329496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a1f09-7506-45b6-8017-8842548ee0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:05.330117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a1f09-7506-45b6-8017-8842548ee0dd admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:05.330388 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a1f09-7506-45b6-8017-8842548ee0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:05.330653 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a1f09-7506-45b6-8017-8842548ee0dd admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:05.331340 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a1f09-7506-45b6-8017-8842548ee0dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:05.333313 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a1f09-7506-45b6-8017-8842548ee0dd admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:05.372739 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1202/2417] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:00:05 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:00:05.379383 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d54653-d427-48fb-bd89-c2963f9d2cf6 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:05.397979 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d54653-d427-48fb-bd89-c2963f9d2cf6 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:05.436205 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:05.474589 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1156f1b-948a-42b5-98a2-ca3f6a69ca18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:05.475296 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1156f1b-948a-42b5-98a2-ca3f6a69ca18 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:05.475558 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1156f1b-948a-42b5-98a2-ca3f6a69ca18 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:05.475831 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1156f1b-948a-42b5-98a2-ca3f6a69ca18 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:05.476503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1156f1b-948a-42b5-98a2-ca3f6a69ca18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:05.478529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1156f1b-948a-42b5-98a2-ca3f6a69ca18 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:05.523848 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1216/2418] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:00:05 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:00:05.526333 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a1f09-7506-45b6-8017-8842548ee0dd admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:05.540197 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a1f09-7506-45b6-8017-8842548ee0dd admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:05.549094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80502f4f-1232-48da-9388-0f4118b0e30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:05.549755 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80502f4f-1232-48da-9388-0f4118b0e30f admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:05.550038 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80502f4f-1232-48da-9388-0f4118b0e30f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:05.550307 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80502f4f-1232-48da-9388-0f4118b0e30f admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:05.554900 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80502f4f-1232-48da-9388-0f4118b0e30f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:05.554900 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80502f4f-1232-48da-9388-0f4118b0e30f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:05.565160 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1203/2419] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:00:05.571653 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1156f1b-948a-42b5-98a2-ca3f6a69ca18 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:05.583322 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1156f1b-948a-42b5-98a2-ca3f6a69ca18 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:05.589978 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80bdaf-e904-4b5f-aa05-7312510f6695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:05.590626 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80bdaf-e904-4b5f-aa05-7312510f6695 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:05.590913 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80bdaf-e904-4b5f-aa05-7312510f6695 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:05.591208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80bdaf-e904-4b5f-aa05-7312510f6695 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:05.591871 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80bdaf-e904-4b5f-aa05-7312510f6695 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:05.594361 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80bdaf-e904-4b5f-aa05-7312510f6695 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:05.611008 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1217/2420] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:00:05.615570 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80502f4f-1232-48da-9388-0f4118b0e30f admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:05.629082 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80502f4f-1232-48da-9388-0f4118b0e30f admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:05.636165 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8de1db-b5d7-4c06-bf3a-b7c14890fbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:05.637121 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8de1db-b5d7-4c06-bf3a-b7c14890fbe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:05.637344 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8de1db-b5d7-4c06-bf3a-b7c14890fbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:05.637615 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8de1db-b5d7-4c06-bf3a-b7c14890fbe1 admin admin] PATH_INFO: `/v3/projects/176afd6723734b4b942402cffaf2eb9b/users/1d535834e94c4ec29bf3b7b1fe617982/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:05.656472 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8de1db-b5d7-4c06-bf3a-b7c14890fbe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=176afd6723734b4b942402cffaf2eb9b, user_id=1d535834e94c4ec29bf3b7b1fe617982, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:05.659518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8de1db-b5d7-4c06-bf3a-b7c14890fbe1 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:05.677848 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1204/2421] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:00:05 2022] PUT /identity/v3/projects/176afd6723734b4b942402cffaf2eb9b/users/1d535834e94c4ec29bf3b7b1fe617982/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:05.689688 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75787831-9ada-450a-8432-e1808aea265d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:05.690054 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75787831-9ada-450a-8432-e1808aea265d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:05.690490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75787831-9ada-450a-8432-e1808aea265d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:05.736134 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-75787831-9ada-450a-8432-e1808aea265d None None] MFA Rules not processed for user `1d535834e94c4ec29bf3b7b1fe617982`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:05.737403 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-75787831-9ada-450a-8432-e1808aea265d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:05.854391 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1218/2422] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:05.869636 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8de1db-b5d7-4c06-bf3a-b7c14890fbe1 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:05.871580 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ba8de1db-b5d7-4c06-bf3a-b7c14890fbe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:05.960685 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8de1db-b5d7-4c06-bf3a-b7c14890fbe1 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:05.966685 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa9c13-8e9c-47dd-bbe9-58891d7d6736 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:05.967760 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa9c13-8e9c-47dd-bbe9-58891d7d6736 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:05.968306 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa9c13-8e9c-47dd-bbe9-58891d7d6736 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:05.969038 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa9c13-8e9c-47dd-bbe9-58891d7d6736 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:05.980524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa9c13-8e9c-47dd-bbe9-58891d7d6736 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:05.982228 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa9c13-8e9c-47dd-bbe9-58891d7d6736 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:05.993835 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1205/2423] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:00:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 09:00:08.448393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75787831-9ada-450a-8432-e1808aea265d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:08.451945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-75787831-9ada-450a-8432-e1808aea265d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:08.515766 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75787831-9ada-450a-8432-e1808aea265d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:08.519541 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf512a-6d8a-4239-884f-b5c1e58bfe3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:08.520166 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf512a-6d8a-4239-884f-b5c1e58bfe3e service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:08.520469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf512a-6d8a-4239-884f-b5c1e58bfe3e service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:08.520762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf512a-6d8a-4239-884f-b5c1e58bfe3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:08.522723 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e0cf512a-6d8a-4239-884f-b5c1e58bfe3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:08.582245 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:08.598733 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf512a-6d8a-4239-884f-b5c1e58bfe3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:08.600272 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf512a-6d8a-4239-884f-b5c1e58bfe3e service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:08.610536 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1219/2424] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:00:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:00:08.742186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa9c13-8e9c-47dd-bbe9-58891d7d6736 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:08.757078 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa9c13-8e9c-47dd-bbe9-58891d7d6736 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:08.764215 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fab3b5-433a-4456-a1b2-84479efb09f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:08.766169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fab3b5-433a-4456-a1b2-84479efb09f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:08.766845 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fab3b5-433a-4456-a1b2-84479efb09f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:08.767481 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fab3b5-433a-4456-a1b2-84479efb09f6 admin admin] PATH_INFO: `/v3/users/1d535834e94c4ec29bf3b7b1fe617982` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:08.769141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fab3b5-433a-4456-a1b2-84479efb09f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d535834e94c4ec29bf3b7b1fe617982)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:08.772112 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fab3b5-433a-4456-a1b2-84479efb09f6 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:08.845387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-41fab3b5-433a-4456-a1b2-84479efb09f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d535834e94c4ec29bf3b7b1fe617982'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:08.857744 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-41fab3b5-433a-4456-a1b2-84479efb09f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d535834e94c4ec29bf3b7b1fe617982'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:08.865619 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:08.868307 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-41fab3b5-433a-4456-a1b2-84479efb09f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d535834e94c4ec29bf3b7b1fe617982'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:08.870060 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-41fab3b5-433a-4456-a1b2-84479efb09f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d535834e94c4ec29bf3b7b1fe617982'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:08.870911 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-41fab3b5-433a-4456-a1b2-84479efb09f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d535834e94c4ec29bf3b7b1fe617982'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:08.904374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1206/2425] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:00:08 2022] DELETE /identity/v3/users/1d535834e94c4ec29bf3b7b1fe617982 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:08.917693 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd952d1-bf94-4be5-ab61-a010f650d691 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:08.917693 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd952d1-bf94-4be5-ab61-a010f650d691 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:08.917693 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd952d1-bf94-4be5-ab61-a010f650d691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:08.953796 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ffd952d1-bf94-4be5-ab61-a010f650d691 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:08.955764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ffd952d1-bf94-4be5-ab61-a010f650d691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:09.062109 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1220/2426] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 09:00:09.069962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fab3b5-433a-4456-a1b2-84479efb09f6 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:09.071613 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-41fab3b5-433a-4456-a1b2-84479efb09f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:09.156777 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fab3b5-433a-4456-a1b2-84479efb09f6 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:09.163473 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a3a23-d9ec-46e4-8a8c-cc7887348c75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:09.164542 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a3a23-d9ec-46e4-8a8c-cc7887348c75 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:09.165021 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a3a23-d9ec-46e4-8a8c-cc7887348c75 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:09.165491 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a3a23-d9ec-46e4-8a8c-cc7887348c75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:09.183030 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a3a23-d9ec-46e4-8a8c-cc7887348c75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:09.185847 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a3a23-d9ec-46e4-8a8c-cc7887348c75 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:09.205149 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1207/2427] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:00:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:00:09.589305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd952d1-bf94-4be5-ab61-a010f650d691 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:09.591099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ffd952d1-bf94-4be5-ab61-a010f650d691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:09.604470 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd952d1-bf94-4be5-ab61-a010f650d691 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:09.610492 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7db423-d879-4e30-9022-290ff17fa8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:09.611255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7db423-d879-4e30-9022-290ff17fa8f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:09.611581 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7db423-d879-4e30-9022-290ff17fa8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:09.611857 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7db423-d879-4e30-9022-290ff17fa8f3 admin admin] PATH_INFO: `/v3/projects/176afd6723734b4b942402cffaf2eb9b` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:09.613065 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7db423-d879-4e30-9022-290ff17fa8f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=176afd6723734b4b942402cffaf2eb9b)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:09.616077 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7db423-d879-4e30-9022-290ff17fa8f3 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:09.679607 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-df7db423-d879-4e30-9022-290ff17fa8f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '176afd6723734b4b942402cffaf2eb9b', 'request_id': 'req-df7db423-d879-4e30-9022-290ff17fa8f3'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:09.686127 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-df7db423-d879-4e30-9022-290ff17fa8f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '176afd6723734b4b942402cffaf2eb9b', 'request_id': 'req-df7db423-d879-4e30-9022-290ff17fa8f3'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:09.695254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-df7db423-d879-4e30-9022-290ff17fa8f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '176afd6723734b4b942402cffaf2eb9b', 'request_id': 'req-df7db423-d879-4e30-9022-290ff17fa8f3'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:09.707855 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-df7db423-d879-4e30-9022-290ff17fa8f3 admin admin] The token cache is being invalidate because project 176afd6723734b4b942402cffaf2eb9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:00:09.708143 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-df7db423-d879-4e30-9022-290ff17fa8f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:09.708822 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-df7db423-d879-4e30-9022-290ff17fa8f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:09.710528 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1221/2428] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:00:09 2022] DELETE /identity/v3/projects/176afd6723734b4b942402cffaf2eb9b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:09.715560 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78287c45-c411-4f2a-b960-de1e0ce4595d None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:09.715831 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78287c45-c411-4f2a-b960-de1e0ce4595d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:09.716103 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78287c45-c411-4f2a-b960-de1e0ce4595d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:09.741685 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-78287c45-c411-4f2a-b960-de1e0ce4595d None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:09.743020 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-78287c45-c411-4f2a-b960-de1e0ce4595d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:09.836095 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1208/2429] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:09.848057 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960268d1-95ca-4771-91a3-7e9852b6f58a None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:09.848392 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960268d1-95ca-4771-91a3-7e9852b6f58a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:09.848634 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960268d1-95ca-4771-91a3-7e9852b6f58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:09.879835 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-960268d1-95ca-4771-91a3-7e9852b6f58a None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:09.881125 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-960268d1-95ca-4771-91a3-7e9852b6f58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:09.897132 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1222/2430] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:09.899187 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78287c45-c411-4f2a-b960-de1e0ce4595d None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:09.911286 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78287c45-c411-4f2a-b960-de1e0ce4595d None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:09.919064 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b0a16-dd92-4f24-9944-28dfc51dcad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:09.919893 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b0a16-dd92-4f24-9944-28dfc51dcad1 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:09.920400 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b0a16-dd92-4f24-9944-28dfc51dcad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:09.920798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b0a16-dd92-4f24-9944-28dfc51dcad1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:09.921924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b0a16-dd92-4f24-9944-28dfc51dcad1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 09:00:09.922439 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b0a16-dd92-4f24-9944-28dfc51dcad1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:09.925858 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b0a16-dd92-4f24-9944-28dfc51dcad1 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:09.947387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1209/2431] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 09:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 09:00:09.952851 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960268d1-95ca-4771-91a3-7e9852b6f58a None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:09.966361 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960268d1-95ca-4771-91a3-7e9852b6f58a None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:09.986523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908229b-779c-4695-9b8b-6a004af7867b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:09.987484 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908229b-779c-4695-9b8b-6a004af7867b admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:09.987954 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908229b-779c-4695-9b8b-6a004af7867b admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:09.988078 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908229b-779c-4695-9b8b-6a004af7867b admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:09.988720 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c908229b-779c-4695-9b8b-6a004af7867b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:09.990741 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c908229b-779c-4695-9b8b-6a004af7867b admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:10.030768 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1223/2432] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:00:09 2022] POST /identity/v3/projects => generated 391 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 09:00:10.037783 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b0a16-dd92-4f24-9944-28dfc51dcad1 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:10.056582 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b0a16-dd92-4f24-9944-28dfc51dcad1 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:10.169250 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6ffed-27d1-44a8-8410-382c37343851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:10.169250 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6ffed-27d1-44a8-8410-382c37343851 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:10.169250 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6ffed-27d1-44a8-8410-382c37343851 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:10.170266 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa6ffed-27d1-44a8-8410-382c37343851 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:10.171642 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa6ffed-27d1-44a8-8410-382c37343851 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:10.176064 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa6ffed-27d1-44a8-8410-382c37343851 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:10.193652 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:10.225486 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1210/2433] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:00:10 2022] POST /identity/v3/users => generated 296 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:00:10.227328 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908229b-779c-4695-9b8b-6a004af7867b admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:10.240517 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908229b-779c-4695-9b8b-6a004af7867b admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:10.248211 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a86d59e-fc9f-4489-b552-54924f6c4c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:10.249644 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86d59e-fc9f-4489-b552-54924f6c4c3f admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:10.250203 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86d59e-fc9f-4489-b552-54924f6c4c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:10.250768 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a86d59e-fc9f-4489-b552-54924f6c4c3f admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:10.252147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a86d59e-fc9f-4489-b552-54924f6c4c3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:10.256111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a86d59e-fc9f-4489-b552-54924f6c4c3f admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:10.274503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1224/2434] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 09:00:10.280625 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6ffed-27d1-44a8-8410-382c37343851 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:10.292766 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa6ffed-27d1-44a8-8410-382c37343851 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:10.298319 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7455acc-dda4-4899-972d-2b3a3ccaa947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:10.298991 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7455acc-dda4-4899-972d-2b3a3ccaa947 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:10.299277 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7455acc-dda4-4899-972d-2b3a3ccaa947 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:10.299550 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7455acc-dda4-4899-972d-2b3a3ccaa947 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:10.300222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7455acc-dda4-4899-972d-2b3a3ccaa947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:10.301897 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7455acc-dda4-4899-972d-2b3a3ccaa947 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:10.330434 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1211/2435] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 09:00:10.335471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a86d59e-fc9f-4489-b552-54924f6c4c3f admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:10.362919 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a86d59e-fc9f-4489-b552-54924f6c4c3f admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:10.384113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b5694c-76da-4e2b-9236-318acde880c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:10.385263 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5694c-76da-4e2b-9236-318acde880c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:10.385764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5694c-76da-4e2b-9236-318acde880c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:10.386262 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5694c-76da-4e2b-9236-318acde880c7 admin admin] PATH_INFO: `/v3/projects/04288ea1b9dc446aa1314fa73b0509cb/users/c53ae5fd033e40c49c591569593ba446/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:10.409899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b5694c-76da-4e2b-9236-318acde880c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04288ea1b9dc446aa1314fa73b0509cb, user_id=c53ae5fd033e40c49c591569593ba446, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:10.416489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b5694c-76da-4e2b-9236-318acde880c7 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:10.448562 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1225/2436] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:00:10 2022] PUT /identity/v3/projects/04288ea1b9dc446aa1314fa73b0509cb/users/c53ae5fd033e40c49c591569593ba446/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:10.462598 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812320d-c519-48e5-a231-73f4624498f6 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:10.462938 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812320d-c519-48e5-a231-73f4624498f6 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:10.463236 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812320d-c519-48e5-a231-73f4624498f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:10.500993 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-4812320d-c519-48e5-a231-73f4624498f6 None None] MFA Rules not processed for user `c53ae5fd033e40c49c591569593ba446`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:10.502569 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4812320d-c519-48e5-a231-73f4624498f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:10.619986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1212/2437] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:10 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:10.635657 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b5694c-76da-4e2b-9236-318acde880c7 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:10.638219 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-75b5694c-76da-4e2b-9236-318acde880c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:10.663486 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:10.727990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b5694c-76da-4e2b-9236-318acde880c7 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:10.732187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223cfc50-01a4-493d-873b-23ed57657c92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:10.732822 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cfc50-01a4-493d-873b-23ed57657c92 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:10.733101 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cfc50-01a4-493d-873b-23ed57657c92 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:10.733413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cfc50-01a4-493d-873b-23ed57657c92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:10.745000 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223cfc50-01a4-493d-873b-23ed57657c92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:10.747220 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223cfc50-01a4-493d-873b-23ed57657c92 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:10.771201 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1226/2438] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:00:10 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:00:14.096545 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812320d-c519-48e5-a231-73f4624498f6 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:14.099514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4812320d-c519-48e5-a231-73f4624498f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:14.203801 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812320d-c519-48e5-a231-73f4624498f6 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:14.211917 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c60ce3e-747c-4955-b9fe-2e55ed5860dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:14.213013 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c60ce3e-747c-4955-b9fe-2e55ed5860dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:14.213476 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c60ce3e-747c-4955-b9fe-2e55ed5860dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:14.213976 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c60ce3e-747c-4955-b9fe-2e55ed5860dc admin admin] PATH_INFO: `/v3/users/b6f722499abc409a8a345bcb70114eaf` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:14.218810 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c60ce3e-747c-4955-b9fe-2e55ed5860dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6f722499abc409a8a345bcb70114eaf)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:14.221996 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c60ce3e-747c-4955-b9fe-2e55ed5860dc admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:14.322779 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-9c60ce3e-747c-4955-b9fe-2e55ed5860dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6f722499abc409a8a345bcb70114eaf'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:14.343853 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-9c60ce3e-747c-4955-b9fe-2e55ed5860dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6f722499abc409a8a345bcb70114eaf'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:14.354807 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-9c60ce3e-747c-4955-b9fe-2e55ed5860dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6f722499abc409a8a345bcb70114eaf'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:14.355596 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-9c60ce3e-747c-4955-b9fe-2e55ed5860dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6f722499abc409a8a345bcb70114eaf'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:14.356230 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-9c60ce3e-747c-4955-b9fe-2e55ed5860dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6f722499abc409a8a345bcb70114eaf'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:14.373607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:14.390207 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1213/2439] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:00:14 2022] DELETE /identity/v3/users/b6f722499abc409a8a345bcb70114eaf => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 09:00:14.652829 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223cfc50-01a4-493d-873b-23ed57657c92 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:14.673009 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223cfc50-01a4-493d-873b-23ed57657c92 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:14.794618 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7cb73-2eb1-4ea0-ab6a-f07a183948f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:14.796618 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7cb73-2eb1-4ea0-ab6a-f07a183948f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:14.797573 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7cb73-2eb1-4ea0-ab6a-f07a183948f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:14.798300 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7cb73-2eb1-4ea0-ab6a-f07a183948f9 admin admin] PATH_INFO: `/v3/projects/88aa61636dd846f0a95be974b6b99d05` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:14.801381 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b7cb73-2eb1-4ea0-ab6a-f07a183948f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88aa61636dd846f0a95be974b6b99d05)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:14.805363 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b7cb73-2eb1-4ea0-ab6a-f07a183948f9 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:14.887159 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f8b7cb73-2eb1-4ea0-ab6a-f07a183948f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88aa61636dd846f0a95be974b6b99d05', 'request_id': 'req-f8b7cb73-2eb1-4ea0-ab6a-f07a183948f9'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:14.897084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f8b7cb73-2eb1-4ea0-ab6a-f07a183948f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88aa61636dd846f0a95be974b6b99d05', 'request_id': 'req-f8b7cb73-2eb1-4ea0-ab6a-f07a183948f9'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:14.912025 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f8b7cb73-2eb1-4ea0-ab6a-f07a183948f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88aa61636dd846f0a95be974b6b99d05', 'request_id': 'req-f8b7cb73-2eb1-4ea0-ab6a-f07a183948f9'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:14.932921 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f8b7cb73-2eb1-4ea0-ab6a-f07a183948f9 admin admin] The token cache is being invalidate because project 88aa61636dd846f0a95be974b6b99d05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:00:14.933220 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f8b7cb73-2eb1-4ea0-ab6a-f07a183948f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:14.933979 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f8b7cb73-2eb1-4ea0-ab6a-f07a183948f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:14.935609 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1227/2440] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:00:14 2022] DELETE /identity/v3/projects/88aa61636dd846f0a95be974b6b99d05 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 09:00:14.943052 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493aa8c0-099f-4942-b679-a0bea327e01f None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:14.943355 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493aa8c0-099f-4942-b679-a0bea327e01f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:14.943591 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493aa8c0-099f-4942-b679-a0bea327e01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:14.976932 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-493aa8c0-099f-4942-b679-a0bea327e01f None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:14.978698 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-493aa8c0-099f-4942-b679-a0bea327e01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:15.086057 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1214/2441] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:15.088126 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f12228-6469-4533-83a0-6ac88d0c21be None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:15.088530 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f12228-6469-4533-83a0-6ac88d0c21be None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:15.089057 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f12228-6469-4533-83a0-6ac88d0c21be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:15.115700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-c6f12228-6469-4533-83a0-6ac88d0c21be None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:15.118212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c6f12228-6469-4533-83a0-6ac88d0c21be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:15.133789 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1228/2442] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:15.139808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493aa8c0-099f-4942-b679-a0bea327e01f None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:15.157989 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493aa8c0-099f-4942-b679-a0bea327e01f None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:15.166832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890c78c-b16f-4c8a-81f3-d1f04108217d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:15.167914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890c78c-b16f-4c8a-81f3-d1f04108217d admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:15.168467 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890c78c-b16f-4c8a-81f3-d1f04108217d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:15.169117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890c78c-b16f-4c8a-81f3-d1f04108217d admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:15.170197 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f890c78c-b16f-4c8a-81f3-d1f04108217d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 09:00:15.170710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f890c78c-b16f-4c8a-81f3-d1f04108217d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:15.174084 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f890c78c-b16f-4c8a-81f3-d1f04108217d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:15.200826 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1215/2443] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 09:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 09:00:15.207238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f12228-6469-4533-83a0-6ac88d0c21be None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:15.218578 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f12228-6469-4533-83a0-6ac88d0c21be None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:15.226381 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d85499-fcfd-48f6-9ac6-2d814833c94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:15.227540 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d85499-fcfd-48f6-9ac6-2d814833c94e admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:15.227944 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d85499-fcfd-48f6-9ac6-2d814833c94e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:15.228403 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d85499-fcfd-48f6-9ac6-2d814833c94e admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:15.229526 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d85499-fcfd-48f6-9ac6-2d814833c94e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:15.233330 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d85499-fcfd-48f6-9ac6-2d814833c94e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:15.285608 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1229/2444] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:00:15 2022] POST /identity/v3/projects => generated 367 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:00:15.287357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890c78c-b16f-4c8a-81f3-d1f04108217d admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:15.305458 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890c78c-b16f-4c8a-81f3-d1f04108217d admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:15.400014 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1f9bd-d076-4bde-9ab0-43bcd9aef335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:15.401109 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1f9bd-d076-4bde-9ab0-43bcd9aef335 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:15.401589 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1f9bd-d076-4bde-9ab0-43bcd9aef335 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:15.401893 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1f9bd-d076-4bde-9ab0-43bcd9aef335 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:15.402627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f1f9bd-d076-4bde-9ab0-43bcd9aef335 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:15.406004 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f1f9bd-d076-4bde-9ab0-43bcd9aef335 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:15.439208 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1216/2445] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:00:15 2022] POST /identity/v3/users => generated 284 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:00:15.445354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d85499-fcfd-48f6-9ac6-2d814833c94e admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:15.462318 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d85499-fcfd-48f6-9ac6-2d814833c94e admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:15.470708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d999a02-6e4f-41b9-a1ce-48ce121fec13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:15.472413 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d999a02-6e4f-41b9-a1ce-48ce121fec13 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:15.473028 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d999a02-6e4f-41b9-a1ce-48ce121fec13 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:15.473690 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d999a02-6e4f-41b9-a1ce-48ce121fec13 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:15.474966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d999a02-6e4f-41b9-a1ce-48ce121fec13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:15.477962 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d999a02-6e4f-41b9-a1ce-48ce121fec13 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:15.487102 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:15.500375 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1f9bd-d076-4bde-9ab0-43bcd9aef335 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:15.501014 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1230/2446] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:00:15.516894 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1f9bd-d076-4bde-9ab0-43bcd9aef335 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:15.527174 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5c0e2-a9e7-4bd7-a046-8e5f14184082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:15.528132 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5c0e2-a9e7-4bd7-a046-8e5f14184082 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:15.528619 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5c0e2-a9e7-4bd7-a046-8e5f14184082 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:15.529080 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5c0e2-a9e7-4bd7-a046-8e5f14184082 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:15.530051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5c0e2-a9e7-4bd7-a046-8e5f14184082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:15.532796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5c0e2-a9e7-4bd7-a046-8e5f14184082 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:15.549503 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1217/2447] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 09:00:15.555469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d999a02-6e4f-41b9-a1ce-48ce121fec13 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:15.572956 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d999a02-6e4f-41b9-a1ce-48ce121fec13 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:15.586677 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc6f61-097e-4b80-9ae7-70b9d42c00d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:15.587928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc6f61-097e-4b80-9ae7-70b9d42c00d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:15.588392 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc6f61-097e-4b80-9ae7-70b9d42c00d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:15.588858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc6f61-097e-4b80-9ae7-70b9d42c00d2 admin admin] PATH_INFO: `/v3/projects/c1e94046a72147078d483cefe831625e/users/ed7f808e6cb24764bdc95bd160a4a5e0/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:15.620147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc6f61-097e-4b80-9ae7-70b9d42c00d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1e94046a72147078d483cefe831625e, user_id=ed7f808e6cb24764bdc95bd160a4a5e0, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:15.626511 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc6f61-097e-4b80-9ae7-70b9d42c00d2 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:15.660103 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1231/2448] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:00:15 2022] PUT /identity/v3/projects/c1e94046a72147078d483cefe831625e/users/ed7f808e6cb24764bdc95bd160a4a5e0/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:15.673727 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb7d66-0b83-4b8b-8a73-56af3bb839b2 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:15.674311 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb7d66-0b83-4b8b-8a73-56af3bb839b2 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:15.674311 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb7d66-0b83-4b8b-8a73-56af3bb839b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:15.718630 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-31bb7d66-0b83-4b8b-8a73-56af3bb839b2 None None] MFA Rules not processed for user `ed7f808e6cb24764bdc95bd160a4a5e0`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:15.720228 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-31bb7d66-0b83-4b8b-8a73-56af3bb839b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:15.803888 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:15.840530 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1218/2449] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:15.856387 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc6f61-097e-4b80-9ae7-70b9d42c00d2 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:15.858466 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-aadc6f61-097e-4b80-9ae7-70b9d42c00d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:15.980322 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc6f61-097e-4b80-9ae7-70b9d42c00d2 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:15.986702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841253ec-ffa5-4ed2-a615-5c9e5bf91c03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:15.987928 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841253ec-ffa5-4ed2-a615-5c9e5bf91c03 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:15.988344 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841253ec-ffa5-4ed2-a615-5c9e5bf91c03 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:15.988842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841253ec-ffa5-4ed2-a615-5c9e5bf91c03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:16.006591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841253ec-ffa5-4ed2-a615-5c9e5bf91c03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:16.008392 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841253ec-ffa5-4ed2-a615-5c9e5bf91c03 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:16.029651 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1232/2450] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:00:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:00:20.433367 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb7d66-0b83-4b8b-8a73-56af3bb839b2 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:20.435712 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-31bb7d66-0b83-4b8b-8a73-56af3bb839b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:20.557216 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb7d66-0b83-4b8b-8a73-56af3bb839b2 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:20.562074 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5d213-03a1-4967-b171-98274f7740b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:20.562810 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5d213-03a1-4967-b171-98274f7740b3 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:20.563120 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5d213-03a1-4967-b171-98274f7740b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:20.563397 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5d213-03a1-4967-b171-98274f7740b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:20.566448 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8ef5d213-03a1-4967-b171-98274f7740b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:20.651914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef5d213-03a1-4967-b171-98274f7740b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:20.653732 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef5d213-03a1-4967-b171-98274f7740b3 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:20.667322 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1219/2451] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:00:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:00:26.014804 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=36,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:30.661238 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=53 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:31.649708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841253ec-ffa5-4ed2-a615-5c9e5bf91c03 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:31.651867 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-841253ec-ffa5-4ed2-a615-5c9e5bf91c03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:31.669322 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841253ec-ffa5-4ed2-a615-5c9e5bf91c03 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:31.676770 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53be28-19ac-48cb-bd31-c8c0470aae85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:31.677532 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53be28-19ac-48cb-bd31-c8c0470aae85 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:31.677914 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53be28-19ac-48cb-bd31-c8c0470aae85 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:31.678300 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53be28-19ac-48cb-bd31-c8c0470aae85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:31.680612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1d53be28-19ac-48cb-bd31-c8c0470aae85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:31.781709 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d53be28-19ac-48cb-bd31-c8c0470aae85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:31.784515 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d53be28-19ac-48cb-bd31-c8c0470aae85 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:31.831858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1233/2452] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:00:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:00:33.031979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5d213-03a1-4967-b171-98274f7740b3 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:33.053521 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5d213-03a1-4967-b171-98274f7740b3 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:33.067518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b955228-95ab-4497-9b85-b7ad062dc3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:33.068614 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b955228-95ab-4497-9b85-b7ad062dc3be admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:33.069076 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b955228-95ab-4497-9b85-b7ad062dc3be admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:33.069567 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b955228-95ab-4497-9b85-b7ad062dc3be admin admin] PATH_INFO: `/v3/users/ed7f808e6cb24764bdc95bd160a4a5e0` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:33.071795 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b955228-95ab-4497-9b85-b7ad062dc3be admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed7f808e6cb24764bdc95bd160a4a5e0)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:33.076032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b955228-95ab-4497-9b85-b7ad062dc3be admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:33.193273 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-2b955228-95ab-4497-9b85-b7ad062dc3be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed7f808e6cb24764bdc95bd160a4a5e0'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:33.221034 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-2b955228-95ab-4497-9b85-b7ad062dc3be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed7f808e6cb24764bdc95bd160a4a5e0'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:33.234634 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-2b955228-95ab-4497-9b85-b7ad062dc3be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed7f808e6cb24764bdc95bd160a4a5e0'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:33.236251 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-2b955228-95ab-4497-9b85-b7ad062dc3be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed7f808e6cb24764bdc95bd160a4a5e0'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:33.237119 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-2b955228-95ab-4497-9b85-b7ad062dc3be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed7f808e6cb24764bdc95bd160a4a5e0'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:33.267674 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1220/2453] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:00:33 2022] DELETE /identity/v3/users/ed7f808e6cb24764bdc95bd160a4a5e0 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:33.273347 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1491e85-5a37-4067-a1bd-37dee479d9fc None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:33.273982 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1491e85-5a37-4067-a1bd-37dee479d9fc None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:33.273982 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1491e85-5a37-4067-a1bd-37dee479d9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:33.319515 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-c1491e85-5a37-4067-a1bd-37dee479d9fc None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:33.321291 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c1491e85-5a37-4067-a1bd-37dee479d9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:33.428096 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1234/2454] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:33.442741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b955228-95ab-4497-9b85-b7ad062dc3be admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:33.444366 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-2b955228-95ab-4497-9b85-b7ad062dc3be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:33.555655 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b955228-95ab-4497-9b85-b7ad062dc3be admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:33.574036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10685d31-e98c-4463-9c88-9a47b161bf6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:33.575122 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10685d31-e98c-4463-9c88-9a47b161bf6d service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:33.575526 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10685d31-e98c-4463-9c88-9a47b161bf6d service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:33.576023 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10685d31-e98c-4463-9c88-9a47b161bf6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:33.595247 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10685d31-e98c-4463-9c88-9a47b161bf6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:33.596967 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10685d31-e98c-4463-9c88-9a47b161bf6d service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:33.610385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1221/2455] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:00:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 09:00:34.290450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1491e85-5a37-4067-a1bd-37dee479d9fc None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:34.294109 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c1491e85-5a37-4067-a1bd-37dee479d9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:34.311206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10685d31-e98c-4463-9c88-9a47b161bf6d service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:34.312681 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1491e85-5a37-4067-a1bd-37dee479d9fc None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:34.315780 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-10685d31-e98c-4463-9c88-9a47b161bf6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:34.320629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc8440-3f8d-4c3c-8a40-1b6012af78b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:34.322051 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc8440-3f8d-4c3c-8a40-1b6012af78b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:34.322929 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc8440-3f8d-4c3c-8a40-1b6012af78b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:34.323673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc8440-3f8d-4c3c-8a40-1b6012af78b0 admin admin] PATH_INFO: `/v3/users/c53ae5fd033e40c49c591569593ba446` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:34.325634 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc8440-3f8d-4c3c-8a40-1b6012af78b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c53ae5fd033e40c49c591569593ba446)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:34.328436 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc8440-3f8d-4c3c-8a40-1b6012af78b0 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:34.341311 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10685d31-e98c-4463-9c88-9a47b161bf6d service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:34.362073 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd329b51-99c4-4e8b-adf9-5da7298a1f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:34.369059 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd329b51-99c4-4e8b-adf9-5da7298a1f07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:34.369425 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd329b51-99c4-4e8b-adf9-5da7298a1f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:34.369879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd329b51-99c4-4e8b-adf9-5da7298a1f07 admin admin] PATH_INFO: `/v3/projects/c1e94046a72147078d483cefe831625e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:34.376941 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd329b51-99c4-4e8b-adf9-5da7298a1f07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1e94046a72147078d483cefe831625e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:34.381340 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd329b51-99c4-4e8b-adf9-5da7298a1f07 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:34.420785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-61fc8440-3f8d-4c3c-8a40-1b6012af78b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c53ae5fd033e40c49c591569593ba446'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:34.443372 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-61fc8440-3f8d-4c3c-8a40-1b6012af78b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c53ae5fd033e40c49c591569593ba446'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:34.457152 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-61fc8440-3f8d-4c3c-8a40-1b6012af78b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c53ae5fd033e40c49c591569593ba446'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:34.469242 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-61fc8440-3f8d-4c3c-8a40-1b6012af78b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c53ae5fd033e40c49c591569593ba446'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:34.470131 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-61fc8440-3f8d-4c3c-8a40-1b6012af78b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c53ae5fd033e40c49c591569593ba446'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:34.473484 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-bd329b51-99c4-4e8b-adf9-5da7298a1f07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1e94046a72147078d483cefe831625e', 'request_id': 'req-bd329b51-99c4-4e8b-adf9-5da7298a1f07'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:34.479099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1235/2456] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:00:34 2022] DELETE /identity/v3/users/c53ae5fd033e40c49c591569593ba446 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:34.487867 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5146016e-4100-4760-8083-db32621380fd None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:34.488199 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5146016e-4100-4760-8083-db32621380fd None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:34.488573 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5146016e-4100-4760-8083-db32621380fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:34.490749 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-bd329b51-99c4-4e8b-adf9-5da7298a1f07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1e94046a72147078d483cefe831625e', 'request_id': 'req-bd329b51-99c4-4e8b-adf9-5da7298a1f07'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:34.503377 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-bd329b51-99c4-4e8b-adf9-5da7298a1f07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1e94046a72147078d483cefe831625e', 'request_id': 'req-bd329b51-99c4-4e8b-adf9-5da7298a1f07'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:34.510190 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:34.516216 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:34.521902 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-5146016e-4100-4760-8083-db32621380fd None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:34.525551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-5146016e-4100-4760-8083-db32621380fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:34.535017 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-bd329b51-99c4-4e8b-adf9-5da7298a1f07 admin admin] The token cache is being invalidate because project c1e94046a72147078d483cefe831625e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:00:34.535344 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-bd329b51-99c4-4e8b-adf9-5da7298a1f07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:34.537668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-bd329b51-99c4-4e8b-adf9-5da7298a1f07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:34.543346 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1222/2457] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:00:34 2022] DELETE /identity/v3/projects/c1e94046a72147078d483cefe831625e => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:34.548320 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76b893-d751-41a5-a4a3-4b35005f3b9e None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:34.548656 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76b893-d751-41a5-a4a3-4b35005f3b9e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:34.548977 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76b893-d751-41a5-a4a3-4b35005f3b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:34.594684 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ca76b893-d751-41a5-a4a3-4b35005f3b9e None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:34.598964 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ca76b893-d751-41a5-a4a3-4b35005f3b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:34.647086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1236/2458] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:34.666817 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5146016e-4100-4760-8083-db32621380fd None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:34.669363 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-5146016e-4100-4760-8083-db32621380fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:34.706073 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1223/2459] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:34.714601 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039c797-d2c7-4ca4-b14c-8796bb05a49e None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:34.715001 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039c797-d2c7-4ca4-b14c-8796bb05a49e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:34.715340 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039c797-d2c7-4ca4-b14c-8796bb05a49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:34.770379 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-7039c797-d2c7-4ca4-b14c-8796bb05a49e None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:34.772928 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7039c797-d2c7-4ca4-b14c-8796bb05a49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:34.782103 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5146016e-4100-4760-8083-db32621380fd None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:34.797086 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1224/2460] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:34.798604 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039c797-d2c7-4ca4-b14c-8796bb05a49e None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:34.799400 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166ed20-a326-4b3c-9305-c6ce81601fcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:34.800294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166ed20-a326-4b3c-9305-c6ce81601fcf service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:34.800638 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166ed20-a326-4b3c-9305-c6ce81601fcf service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:34.801204 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166ed20-a326-4b3c-9305-c6ce81601fcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:34.814635 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166ed20-a326-4b3c-9305-c6ce81601fcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:34.816440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166ed20-a326-4b3c-9305-c6ce81601fcf service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:34.831323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039c797-d2c7-4ca4-b14c-8796bb05a49e None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:34.833277 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1237/2461] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:00:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 09:00:34.841523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bdfa49-3590-4b42-9f40-8b144e3fa98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:34.842748 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bdfa49-3590-4b42-9f40-8b144e3fa98c admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:34.843203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bdfa49-3590-4b42-9f40-8b144e3fa98c admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:34.843593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bdfa49-3590-4b42-9f40-8b144e3fa98c admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:34.844711 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bdfa49-3590-4b42-9f40-8b144e3fa98c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 09:00:34.845151 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bdfa49-3590-4b42-9f40-8b144e3fa98c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:34.848045 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bdfa49-3590-4b42-9f40-8b144e3fa98c admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:34.862431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1225/2462] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 09:00:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 09:00:34.869203 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166ed20-a326-4b3c-9305-c6ce81601fcf service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:34.885265 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166ed20-a326-4b3c-9305-c6ce81601fcf service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:34.895996 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94cff2-b187-4065-832d-a83173a1de5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:34.897116 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94cff2-b187-4065-832d-a83173a1de5e admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:34.897609 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94cff2-b187-4065-832d-a83173a1de5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:34.898239 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94cff2-b187-4065-832d-a83173a1de5e admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:34.899198 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94cff2-b187-4065-832d-a83173a1de5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:34.902440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94cff2-b187-4065-832d-a83173a1de5e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:34.954690 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1238/2463] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:00:34 2022] POST /identity/v3/projects => generated 385 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:00:34.960189 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bdfa49-3590-4b42-9f40-8b144e3fa98c admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:34.977223 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bdfa49-3590-4b42-9f40-8b144e3fa98c admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:35.094717 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76514d00-27fc-41b4-a501-8d2eddc6507b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:35.095755 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76514d00-27fc-41b4-a501-8d2eddc6507b admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:35.096230 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76514d00-27fc-41b4-a501-8d2eddc6507b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:35.096732 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76514d00-27fc-41b4-a501-8d2eddc6507b admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:35.097821 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76514d00-27fc-41b4-a501-8d2eddc6507b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:35.100319 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76514d00-27fc-41b4-a501-8d2eddc6507b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:35.137012 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1226/2464] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:00:34 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 09:00:35.143280 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94cff2-b187-4065-832d-a83173a1de5e admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:35.159581 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94cff2-b187-4065-832d-a83173a1de5e admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:35.166242 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be33278-fe10-4636-a78b-a82ee2180c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:35.167117 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be33278-fe10-4636-a78b-a82ee2180c1d admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:35.167653 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be33278-fe10-4636-a78b-a82ee2180c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:35.168103 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be33278-fe10-4636-a78b-a82ee2180c1d admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:35.169041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be33278-fe10-4636-a78b-a82ee2180c1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:35.172108 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be33278-fe10-4636-a78b-a82ee2180c1d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:35.188872 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1239/2465] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:00:35.193722 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76514d00-27fc-41b4-a501-8d2eddc6507b admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:35.210305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76514d00-27fc-41b4-a501-8d2eddc6507b admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:35.217898 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224460b3-98ac-44c3-9f30-3f8fa49ad6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:35.218815 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224460b3-98ac-44c3-9f30-3f8fa49ad6ea admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:35.219223 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224460b3-98ac-44c3-9f30-3f8fa49ad6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:35.219642 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224460b3-98ac-44c3-9f30-3f8fa49ad6ea admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:35.220935 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224460b3-98ac-44c3-9f30-3f8fa49ad6ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:35.223131 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224460b3-98ac-44c3-9f30-3f8fa49ad6ea admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:35.239948 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1227/2466] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:00:35.246178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be33278-fe10-4636-a78b-a82ee2180c1d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:35.263571 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be33278-fe10-4636-a78b-a82ee2180c1d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:35.272164 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b32d13-5296-419c-8081-5195fd6a78b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:35.273373 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b32d13-5296-419c-8081-5195fd6a78b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:35.273867 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b32d13-5296-419c-8081-5195fd6a78b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:35.274351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b32d13-5296-419c-8081-5195fd6a78b9 admin admin] PATH_INFO: `/v3/projects/1089734499d54908a2e8c064072ba940/users/313a829827f34359a46cd53d14473179/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:35.303673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b32d13-5296-419c-8081-5195fd6a78b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1089734499d54908a2e8c064072ba940, user_id=313a829827f34359a46cd53d14473179, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:35.308775 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b32d13-5296-419c-8081-5195fd6a78b9 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:35.328938 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1240/2467] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:00:35 2022] PUT /identity/v3/projects/1089734499d54908a2e8c064072ba940/users/313a829827f34359a46cd53d14473179/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:35.346557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c22a28-69a3-4f4b-8674-9a173199ea6f None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:35.347428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c22a28-69a3-4f4b-8674-9a173199ea6f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:35.347842 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c22a28-69a3-4f4b-8674-9a173199ea6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:35.404474 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-92c22a28-69a3-4f4b-8674-9a173199ea6f None None] MFA Rules not processed for user `313a829827f34359a46cd53d14473179`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:35.406216 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-92c22a28-69a3-4f4b-8674-9a173199ea6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:35.533139 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:35.598765 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1228/2468] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 09:00:35.603443 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b32d13-5296-419c-8081-5195fd6a78b9 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:35.705000 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:35.738440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b32d13-5296-419c-8081-5195fd6a78b9 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:35.748412 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3991b26-d652-4aea-abcf-f06782842573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:35.749161 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3991b26-d652-4aea-abcf-f06782842573 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:35.749511 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3991b26-d652-4aea-abcf-f06782842573 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:35.749852 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3991b26-d652-4aea-abcf-f06782842573 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:35.751576 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3991b26-d652-4aea-abcf-f06782842573 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:35.753877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3991b26-d652-4aea-abcf-f06782842573 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:35.816752 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c22a28-69a3-4f4b-8674-9a173199ea6f None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:35.817743 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1241/2469] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:00:35 2022] POST /identity/v3/projects => generated 385 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 09:00:35.835715 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c22a28-69a3-4f4b-8674-9a173199ea6f None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:35.882703 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3991b26-d652-4aea-abcf-f06782842573 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:35.886273 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e3991b26-d652-4aea-abcf-f06782842573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:35.993921 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592aeef5-eba1-4335-a473-7fa5d34904d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:35.994485 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592aeef5-eba1-4335-a473-7fa5d34904d4 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:35.994762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592aeef5-eba1-4335-a473-7fa5d34904d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:35.995088 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592aeef5-eba1-4335-a473-7fa5d34904d4 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:35.995910 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592aeef5-eba1-4335-a473-7fa5d34904d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:35.997903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592aeef5-eba1-4335-a473-7fa5d34904d4 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:36.027305 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3991b26-d652-4aea-abcf-f06782842573 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:36.045145 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1229/2470] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:00:35 2022] POST /identity/v3/users => generated 299 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:00:36.048681 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab89cfa-46f7-4537-bca9-494bf3b5d7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:36.049801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab89cfa-46f7-4537-bca9-494bf3b5d7a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:36.050137 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab89cfa-46f7-4537-bca9-494bf3b5d7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:36.050499 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab89cfa-46f7-4537-bca9-494bf3b5d7a8 admin admin] PATH_INFO: `/v3/projects/04288ea1b9dc446aa1314fa73b0509cb` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:36.052098 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab89cfa-46f7-4537-bca9-494bf3b5d7a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04288ea1b9dc446aa1314fa73b0509cb)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:36.054490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab89cfa-46f7-4537-bca9-494bf3b5d7a8 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:36.057138 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592aeef5-eba1-4335-a473-7fa5d34904d4 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:36.087630 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592aeef5-eba1-4335-a473-7fa5d34904d4 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:36.099609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eacb139-6604-4bc9-80e0-0579ebf764bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:36.100919 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eacb139-6604-4bc9-80e0-0579ebf764bb admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:36.101627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eacb139-6604-4bc9-80e0-0579ebf764bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:36.102203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eacb139-6604-4bc9-80e0-0579ebf764bb admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:36.103490 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eacb139-6604-4bc9-80e0-0579ebf764bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:36.110199 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eacb139-6604-4bc9-80e0-0579ebf764bb admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:36.145409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2ab89cfa-46f7-4537-bca9-494bf3b5d7a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04288ea1b9dc446aa1314fa73b0509cb', 'request_id': 'req-2ab89cfa-46f7-4537-bca9-494bf3b5d7a8'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:36.145749 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1230/2471] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:00:36.146163 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eacb139-6604-4bc9-80e0-0579ebf764bb admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:36.158437 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2ab89cfa-46f7-4537-bca9-494bf3b5d7a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04288ea1b9dc446aa1314fa73b0509cb', 'request_id': 'req-2ab89cfa-46f7-4537-bca9-494bf3b5d7a8'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:36.164614 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eacb139-6604-4bc9-80e0-0579ebf764bb admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:36.169395 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2ab89cfa-46f7-4537-bca9-494bf3b5d7a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04288ea1b9dc446aa1314fa73b0509cb', 'request_id': 'req-2ab89cfa-46f7-4537-bca9-494bf3b5d7a8'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:36.189957 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2ab89cfa-46f7-4537-bca9-494bf3b5d7a8 admin admin] The token cache is being invalidate because project 04288ea1b9dc446aa1314fa73b0509cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:00:36.190558 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2ab89cfa-46f7-4537-bca9-494bf3b5d7a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:36.191386 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2ab89cfa-46f7-4537-bca9-494bf3b5d7a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:36.193726 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1242/2472] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:00:35 2022] DELETE /identity/v3/projects/04288ea1b9dc446aa1314fa73b0509cb => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 09:00:36.288676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a1618-289c-4cdc-a0af-1aac9ac0b174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:36.289731 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a1618-289c-4cdc-a0af-1aac9ac0b174 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:36.290135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a1618-289c-4cdc-a0af-1aac9ac0b174 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:36.290372 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a1618-289c-4cdc-a0af-1aac9ac0b174 admin admin] PATH_INFO: `/v3/projects/b35037360d1a407f98bcd8732bb0f9af/users/311f7bf68b1a45e990bd85564c4e3639/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:36.316681 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a1618-289c-4cdc-a0af-1aac9ac0b174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b35037360d1a407f98bcd8732bb0f9af, user_id=311f7bf68b1a45e990bd85564c4e3639, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:36.319779 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a1618-289c-4cdc-a0af-1aac9ac0b174 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:36.342569 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1231/2473] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:00:36 2022] PUT /identity/v3/projects/b35037360d1a407f98bcd8732bb0f9af/users/311f7bf68b1a45e990bd85564c4e3639/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:36.348769 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab89cfa-46f7-4537-bca9-494bf3b5d7a8 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:36.350536 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2ab89cfa-46f7-4537-bca9-494bf3b5d7a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:36.384051 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab89cfa-46f7-4537-bca9-494bf3b5d7a8 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:36.471418 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85beaa0-3b71-462f-9123-ea1e49d6a145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:36.472256 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85beaa0-3b71-462f-9123-ea1e49d6a145 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:36.472476 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85beaa0-3b71-462f-9123-ea1e49d6a145 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:36.472732 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85beaa0-3b71-462f-9123-ea1e49d6a145 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:36.473420 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85beaa0-3b71-462f-9123-ea1e49d6a145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:36.475292 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85beaa0-3b71-462f-9123-ea1e49d6a145 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:36.491988 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1243/2474] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:00:36.498645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a1618-289c-4cdc-a0af-1aac9ac0b174 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:36.526749 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a1618-289c-4cdc-a0af-1aac9ac0b174 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:36.534479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc7600f-c5f7-4fc8-b6af-e84fef49eca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:36.535964 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7600f-c5f7-4fc8-b6af-e84fef49eca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:36.536604 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7600f-c5f7-4fc8-b6af-e84fef49eca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:36.537174 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7600f-c5f7-4fc8-b6af-e84fef49eca3 admin admin] PATH_INFO: `/v3/projects/b35037360d1a407f98bcd8732bb0f9af/users/311f7bf68b1a45e990bd85564c4e3639/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:36.540503 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc7600f-c5f7-4fc8-b6af-e84fef49eca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b35037360d1a407f98bcd8732bb0f9af, user_id=311f7bf68b1a45e990bd85564c4e3639, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:36.545029 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc7600f-c5f7-4fc8-b6af-e84fef49eca3 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:36.561439 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1232/2475] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:00:36 2022] PUT /identity/v3/projects/b35037360d1a407f98bcd8732bb0f9af/users/311f7bf68b1a45e990bd85564c4e3639/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:36.572880 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4400225-6685-4fad-8326-b6d7ff19402e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:36.573370 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4400225-6685-4fad-8326-b6d7ff19402e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:36.573812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4400225-6685-4fad-8326-b6d7ff19402e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:36.594106 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:36.635225 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-f4400225-6685-4fad-8326-b6d7ff19402e None None] MFA Rules not processed for user `311f7bf68b1a45e990bd85564c4e3639`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:36.637447 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f4400225-6685-4fad-8326-b6d7ff19402e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:36.749417 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1244/2476] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:36 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:36.764468 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc7600f-c5f7-4fc8-b6af-e84fef49eca3 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:36.766605 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4bc7600f-c5f7-4fc8-b6af-e84fef49eca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:36.831741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:36.873077 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc7600f-c5f7-4fc8-b6af-e84fef49eca3 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:36.881294 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26fc2b4-1d67-4241-b53a-b9a1cc58bc71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:36.881994 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26fc2b4-1d67-4241-b53a-b9a1cc58bc71 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:36.882483 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26fc2b4-1d67-4241-b53a-b9a1cc58bc71 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:36.882696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26fc2b4-1d67-4241-b53a-b9a1cc58bc71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:36.900176 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26fc2b4-1d67-4241-b53a-b9a1cc58bc71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:36.902972 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26fc2b4-1d67-4241-b53a-b9a1cc58bc71 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:36.946387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1233/2477] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:00:36 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:00:36.980468 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4400225-6685-4fad-8326-b6d7ff19402e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:37.088860 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4400225-6685-4fad-8326-b6d7ff19402e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:37.096494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5833f-9f6b-44ff-843e-aac5172b36f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:37.097157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5833f-9f6b-44ff-843e-aac5172b36f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:37.097442 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5833f-9f6b-44ff-843e-aac5172b36f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:37.097704 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5833f-9f6b-44ff-843e-aac5172b36f9 admin admin] PATH_INFO: `/v3/users/313a829827f34359a46cd53d14473179` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:37.098880 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5833f-9f6b-44ff-843e-aac5172b36f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=313a829827f34359a46cd53d14473179)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:37.100896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5833f-9f6b-44ff-843e-aac5172b36f9 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:37.190457 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9ba5833f-9f6b-44ff-843e-aac5172b36f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '313a829827f34359a46cd53d14473179'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:37.212299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9ba5833f-9f6b-44ff-843e-aac5172b36f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '313a829827f34359a46cd53d14473179'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:37.224648 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9ba5833f-9f6b-44ff-843e-aac5172b36f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '313a829827f34359a46cd53d14473179'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:37.235082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9ba5833f-9f6b-44ff-843e-aac5172b36f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '313a829827f34359a46cd53d14473179'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:37.242509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9ba5833f-9f6b-44ff-843e-aac5172b36f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '313a829827f34359a46cd53d14473179'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:37.263779 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1245/2478] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:00:36 2022] DELETE /identity/v3/users/313a829827f34359a46cd53d14473179 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:37.267583 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9fcc4-0941-4135-965b-78f2be11f77e None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:37.267752 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9fcc4-0941-4135-965b-78f2be11f77e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:37.268013 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9fcc4-0941-4135-965b-78f2be11f77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:37.293878 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-68d9fcc4-0941-4135-965b-78f2be11f77e None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:37.295186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-68d9fcc4-0941-4135-965b-78f2be11f77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:37.383166 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1234/2479] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:37.396531 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5833f-9f6b-44ff-843e-aac5172b36f9 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:37.398200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-9ba5833f-9f6b-44ff-843e-aac5172b36f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:37.421020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:37.508261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5833f-9f6b-44ff-843e-aac5172b36f9 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:37.514784 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e2f85-2b02-490c-b407-69dce4337dd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:37.516000 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e2f85-2b02-490c-b407-69dce4337dd2 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:37.516512 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e2f85-2b02-490c-b407-69dce4337dd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:37.517010 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e2f85-2b02-490c-b407-69dce4337dd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:37.538865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1e2f85-2b02-490c-b407-69dce4337dd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:37.541732 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1e2f85-2b02-490c-b407-69dce4337dd2 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:37.559090 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1246/2480] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:00:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:00:38.176699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9fcc4-0941-4135-965b-78f2be11f77e None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:38.178968 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-68d9fcc4-0941-4135-965b-78f2be11f77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:38.195848 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9fcc4-0941-4135-965b-78f2be11f77e None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:38.202523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dc64a-3892-4e7a-88ff-06404ef2b2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:38.203335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dc64a-3892-4e7a-88ff-06404ef2b2b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:38.203674 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dc64a-3892-4e7a-88ff-06404ef2b2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:38.204011 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000dc64a-3892-4e7a-88ff-06404ef2b2b9 admin admin] PATH_INFO: `/v3/projects/1089734499d54908a2e8c064072ba940` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:38.205305 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000dc64a-3892-4e7a-88ff-06404ef2b2b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1089734499d54908a2e8c064072ba940)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:38.207368 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000dc64a-3892-4e7a-88ff-06404ef2b2b9 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:38.310790 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-000dc64a-3892-4e7a-88ff-06404ef2b2b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1089734499d54908a2e8c064072ba940', 'request_id': 'req-000dc64a-3892-4e7a-88ff-06404ef2b2b9'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:38.333842 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-000dc64a-3892-4e7a-88ff-06404ef2b2b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1089734499d54908a2e8c064072ba940', 'request_id': 'req-000dc64a-3892-4e7a-88ff-06404ef2b2b9'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:38.345096 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-000dc64a-3892-4e7a-88ff-06404ef2b2b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1089734499d54908a2e8c064072ba940', 'request_id': 'req-000dc64a-3892-4e7a-88ff-06404ef2b2b9'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:38.377642 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-000dc64a-3892-4e7a-88ff-06404ef2b2b9 admin admin] The token cache is being invalidate because project 1089734499d54908a2e8c064072ba940 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:00:38.379838 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-000dc64a-3892-4e7a-88ff-06404ef2b2b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:38.380601 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-000dc64a-3892-4e7a-88ff-06404ef2b2b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:38.382599 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1235/2481] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:00:38 2022] DELETE /identity/v3/projects/1089734499d54908a2e8c064072ba940 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:38.386805 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e2f85-2b02-490c-b407-69dce4337dd2 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:38.399144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e2f85-2b02-490c-b407-69dce4337dd2 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:38.472985 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7b8ec-eb56-45e7-8326-b15bc699ed78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:38.473558 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7b8ec-eb56-45e7-8326-b15bc699ed78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:38.474093 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7b8ec-eb56-45e7-8326-b15bc699ed78 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:38.474568 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7b8ec-eb56-45e7-8326-b15bc699ed78 admin admin] PATH_INFO: `/v3/users/311f7bf68b1a45e990bd85564c4e3639` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:38.476289 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7b8ec-eb56-45e7-8326-b15bc699ed78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=311f7bf68b1a45e990bd85564c4e3639)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:38.478416 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7b8ec-eb56-45e7-8326-b15bc699ed78 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:38.543439 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7ea7b8ec-eb56-45e7-8326-b15bc699ed78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '311f7bf68b1a45e990bd85564c4e3639'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:38.566880 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7ea7b8ec-eb56-45e7-8326-b15bc699ed78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '311f7bf68b1a45e990bd85564c4e3639'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:38.580135 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7ea7b8ec-eb56-45e7-8326-b15bc699ed78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '311f7bf68b1a45e990bd85564c4e3639'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:38.590726 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7ea7b8ec-eb56-45e7-8326-b15bc699ed78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '311f7bf68b1a45e990bd85564c4e3639'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:38.591530 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-7ea7b8ec-eb56-45e7-8326-b15bc699ed78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '311f7bf68b1a45e990bd85564c4e3639'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:38.603837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1247/2482] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:00:38 2022] DELETE /identity/v3/users/311f7bf68b1a45e990bd85564c4e3639 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:39.331110 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dc64a-3892-4e7a-88ff-06404ef2b2b9 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:39.335689 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-000dc64a-3892-4e7a-88ff-06404ef2b2b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:39.354268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000dc64a-3892-4e7a-88ff-06404ef2b2b9 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:39.472270 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37426d0-cff6-4d68-8a6b-e698c0c5c550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:39.473150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37426d0-cff6-4d68-8a6b-e698c0c5c550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:39.473600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37426d0-cff6-4d68-8a6b-e698c0c5c550 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:39.474044 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37426d0-cff6-4d68-8a6b-e698c0c5c550 admin admin] PATH_INFO: `/v3/projects/b35037360d1a407f98bcd8732bb0f9af` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:39.476301 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37426d0-cff6-4d68-8a6b-e698c0c5c550 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b35037360d1a407f98bcd8732bb0f9af)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:39.479407 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37426d0-cff6-4d68-8a6b-e698c0c5c550 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:39.484091 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:39.580952 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b37426d0-cff6-4d68-8a6b-e698c0c5c550 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b35037360d1a407f98bcd8732bb0f9af', 'request_id': 'req-b37426d0-cff6-4d68-8a6b-e698c0c5c550'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:39.606465 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b37426d0-cff6-4d68-8a6b-e698c0c5c550 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b35037360d1a407f98bcd8732bb0f9af', 'request_id': 'req-b37426d0-cff6-4d68-8a6b-e698c0c5c550'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:39.626117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b37426d0-cff6-4d68-8a6b-e698c0c5c550 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b35037360d1a407f98bcd8732bb0f9af', 'request_id': 'req-b37426d0-cff6-4d68-8a6b-e698c0c5c550'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:39.643904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b37426d0-cff6-4d68-8a6b-e698c0c5c550 admin admin] The token cache is being invalidate because project b35037360d1a407f98bcd8732bb0f9af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:00:39.644321 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b37426d0-cff6-4d68-8a6b-e698c0c5c550 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:39.645116 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b37426d0-cff6-4d68-8a6b-e698c0c5c550 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:39.647203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1236/2483] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:00:39 2022] DELETE /identity/v3/projects/b35037360d1a407f98bcd8732bb0f9af => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:39.654371 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20943f-a27b-4adc-8b3a-334eeb83aef5 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:39.654961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20943f-a27b-4adc-8b3a-334eeb83aef5 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:39.654961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20943f-a27b-4adc-8b3a-334eeb83aef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:39.676394 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-fd20943f-a27b-4adc-8b3a-334eeb83aef5 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:39.677648 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-fd20943f-a27b-4adc-8b3a-334eeb83aef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:39.718910 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:39.802733 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1248/2484] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:39.803168 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3c5c9-6454-4c45-9717-3d33ef8b00a7 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:39.803168 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3c5c9-6454-4c45-9717-3d33ef8b00a7 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:39.803168 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3c5c9-6454-4c45-9717-3d33ef8b00a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:39.833608 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-eac3c5c9-6454-4c45-9717-3d33ef8b00a7 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:39.835002 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-eac3c5c9-6454-4c45-9717-3d33ef8b00a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:39.848124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1237/2485] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 09:00:39.854124 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20943f-a27b-4adc-8b3a-334eeb83aef5 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:39.869955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20943f-a27b-4adc-8b3a-334eeb83aef5 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:39.876015 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c2cc9-0d07-4927-b70f-ad2b1a779f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:39.877362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c2cc9-0d07-4927-b70f-ad2b1a779f87 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:39.877362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c2cc9-0d07-4927-b70f-ad2b1a779f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:39.877362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c2cc9-0d07-4927-b70f-ad2b1a779f87 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:39.879021 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c2cc9-0d07-4927-b70f-ad2b1a779f87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 09:00:39.879320 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c2cc9-0d07-4927-b70f-ad2b1a779f87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:39.881184 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c2cc9-0d07-4927-b70f-ad2b1a779f87 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:39.896425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1249/2486] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 09:00:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 09:00:39.901366 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3c5c9-6454-4c45-9717-3d33ef8b00a7 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:39.914350 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3c5c9-6454-4c45-9717-3d33ef8b00a7 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:39.919052 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e674675-509b-4391-984e-b39c481b1d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:39.920245 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e674675-509b-4391-984e-b39c481b1d64 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:39.920831 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e674675-509b-4391-984e-b39c481b1d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:39.921382 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e674675-509b-4391-984e-b39c481b1d64 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:39.922572 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e674675-509b-4391-984e-b39c481b1d64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:39.924807 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e674675-509b-4391-984e-b39c481b1d64 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:39.965986 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c2cc9-0d07-4927-b70f-ad2b1a779f87 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:39.975027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1238/2487] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:00:39 2022] POST /identity/v3/projects => generated 387 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 09:00:39.983029 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c2cc9-0d07-4927-b70f-ad2b1a779f87 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:40.082028 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834fbd5-158d-47a4-b58f-9f19932e8d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:40.082721 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834fbd5-158d-47a4-b58f-9f19932e8d31 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:40.083188 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834fbd5-158d-47a4-b58f-9f19932e8d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:40.083388 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3834fbd5-158d-47a4-b58f-9f19932e8d31 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:40.084566 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3834fbd5-158d-47a4-b58f-9f19932e8d31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:40.087677 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3834fbd5-158d-47a4-b58f-9f19932e8d31 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:40.113334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1250/2488] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:00:39 2022] POST /identity/v3/users => generated 294 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:00:40.117972 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e674675-509b-4391-984e-b39c481b1d64 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:40.130607 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e674675-509b-4391-984e-b39c481b1d64 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:40.136900 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9be86-f15a-49c6-90ef-c30c363e8bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:40.137737 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9be86-f15a-49c6-90ef-c30c363e8bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:40.138176 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9be86-f15a-49c6-90ef-c30c363e8bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:40.138658 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9be86-f15a-49c6-90ef-c30c363e8bf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:40.139901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9be86-f15a-49c6-90ef-c30c363e8bf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:40.142392 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9be86-f15a-49c6-90ef-c30c363e8bf2 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:40.162842 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1239/2489] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 09:00:40.165000 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834fbd5-158d-47a4-b58f-9f19932e8d31 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:40.177835 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3834fbd5-158d-47a4-b58f-9f19932e8d31 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:40.187356 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113bd6f5-053b-4573-8b3e-7fb95cd3dda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:40.188038 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113bd6f5-053b-4573-8b3e-7fb95cd3dda5 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:40.191055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113bd6f5-053b-4573-8b3e-7fb95cd3dda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:40.191055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113bd6f5-053b-4573-8b3e-7fb95cd3dda5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:40.191055 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113bd6f5-053b-4573-8b3e-7fb95cd3dda5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:40.191953 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113bd6f5-053b-4573-8b3e-7fb95cd3dda5 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:40.211867 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1251/2490] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:00:40.217428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9be86-f15a-49c6-90ef-c30c363e8bf2 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:40.245829 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9be86-f15a-49c6-90ef-c30c363e8bf2 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:40.251674 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e94fee3-56c7-4c3b-90be-047d31282e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:40.252577 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94fee3-56c7-4c3b-90be-047d31282e92 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:40.252922 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94fee3-56c7-4c3b-90be-047d31282e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:40.253432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94fee3-56c7-4c3b-90be-047d31282e92 admin admin] PATH_INFO: `/v3/projects/eaf91ca61a914adeab3c2450b8b6191d/users/2a9ee9b775404234b20724380569382c/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:40.272678 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e94fee3-56c7-4c3b-90be-047d31282e92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaf91ca61a914adeab3c2450b8b6191d, user_id=2a9ee9b775404234b20724380569382c, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:40.275722 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e94fee3-56c7-4c3b-90be-047d31282e92 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:40.294951 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1240/2491] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:00:40 2022] PUT /identity/v3/projects/eaf91ca61a914adeab3c2450b8b6191d/users/2a9ee9b775404234b20724380569382c/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:40.307230 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f0737-73ab-4d5f-ad21-1198e3c71d58 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:40.307230 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f0737-73ab-4d5f-ad21-1198e3c71d58 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:40.307578 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f0737-73ab-4d5f-ad21-1198e3c71d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:40.366088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-1a2f0737-73ab-4d5f-ad21-1198e3c71d58 None None] MFA Rules not processed for user `2a9ee9b775404234b20724380569382c`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:40.367405 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1a2f0737-73ab-4d5f-ad21-1198e3c71d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:40.457727 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1252/2492] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:40.462286 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e94fee3-56c7-4c3b-90be-047d31282e92 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:40.550086 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e94fee3-56c7-4c3b-90be-047d31282e92 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:40.558783 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d8097b-37b2-4029-8be1-2b77bec92296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:40.560770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d8097b-37b2-4029-8be1-2b77bec92296 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:40.560770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d8097b-37b2-4029-8be1-2b77bec92296 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:40.560770 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d8097b-37b2-4029-8be1-2b77bec92296 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:40.562103 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d8097b-37b2-4029-8be1-2b77bec92296 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:40.565546 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d8097b-37b2-4029-8be1-2b77bec92296 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:40.611945 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1241/2493] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:00:40 2022] POST /identity/v3/projects => generated 387 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:00:40.613570 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f0737-73ab-4d5f-ad21-1198e3c71d58 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:40.627259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f0737-73ab-4d5f-ad21-1198e3c71d58 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:40.667038 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:40.730028 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc5472-db7d-47bf-812d-011cfbcb2974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:40.730916 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc5472-db7d-47bf-812d-011cfbcb2974 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:40.732941 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc5472-db7d-47bf-812d-011cfbcb2974 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:40.733423 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc5472-db7d-47bf-812d-011cfbcb2974 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:40.734445 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cc5472-db7d-47bf-812d-011cfbcb2974 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:40.737911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cc5472-db7d-47bf-812d-011cfbcb2974 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:40.773905 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1253/2494] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:00:40 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:00:40.778191 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d8097b-37b2-4029-8be1-2b77bec92296 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:40.810937 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d8097b-37b2-4029-8be1-2b77bec92296 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:40.829336 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f83341-14bd-4c4c-9961-4a6d6cbd611e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:40.830315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f83341-14bd-4c4c-9961-4a6d6cbd611e admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:40.830687 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f83341-14bd-4c4c-9961-4a6d6cbd611e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:40.831262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f83341-14bd-4c4c-9961-4a6d6cbd611e admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:40.832103 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f83341-14bd-4c4c-9961-4a6d6cbd611e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:40.834007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f83341-14bd-4c4c-9961-4a6d6cbd611e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:40.848392 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1242/2495] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 09:00:40.852989 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc5472-db7d-47bf-812d-011cfbcb2974 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:40.864852 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc5472-db7d-47bf-812d-011cfbcb2974 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:40.870477 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31082cb-0aa0-4395-997d-9e581578b1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:40.871177 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31082cb-0aa0-4395-997d-9e581578b1d2 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:40.871457 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31082cb-0aa0-4395-997d-9e581578b1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:40.871765 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31082cb-0aa0-4395-997d-9e581578b1d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:40.872406 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31082cb-0aa0-4395-997d-9e581578b1d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:40.874037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31082cb-0aa0-4395-997d-9e581578b1d2 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:40.886556 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1254/2496] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:00:40.891957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f83341-14bd-4c4c-9961-4a6d6cbd611e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:40.898628 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:40.915383 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f83341-14bd-4c4c-9961-4a6d6cbd611e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:40.926051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ba0d2-9346-44a6-a871-1e973eb5af5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:40.927504 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ba0d2-9346-44a6-a871-1e973eb5af5f admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:40.928000 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ba0d2-9346-44a6-a871-1e973eb5af5f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:40.928486 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ba0d2-9346-44a6-a871-1e973eb5af5f admin admin] PATH_INFO: `/v3/projects/49384cd8c94f4098b709b38ed2d2d9f7/users/cbfce778eb7f43da88958b552039dd63/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:40.957404 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ba0d2-9346-44a6-a871-1e973eb5af5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=49384cd8c94f4098b709b38ed2d2d9f7, user_id=cbfce778eb7f43da88958b552039dd63, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:40.963073 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ba0d2-9346-44a6-a871-1e973eb5af5f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:40.986249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1243/2497] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:00:40 2022] PUT /identity/v3/projects/49384cd8c94f4098b709b38ed2d2d9f7/users/cbfce778eb7f43da88958b552039dd63/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:40.997324 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58d976-3133-48f3-9a29-57970d3c1ebc None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:40.997688 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58d976-3133-48f3-9a29-57970d3c1ebc None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:40.998065 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58d976-3133-48f3-9a29-57970d3c1ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:41.040395 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-2a58d976-3133-48f3-9a29-57970d3c1ebc None None] MFA Rules not processed for user `cbfce778eb7f43da88958b552039dd63`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:41.041885 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2a58d976-3133-48f3-9a29-57970d3c1ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:41.177888 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1255/2498] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 09:00:41.183584 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ba0d2-9346-44a6-a871-1e973eb5af5f admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:41.319797 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ba0d2-9346-44a6-a871-1e973eb5af5f admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:41.333005 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c59be62-a35e-4de1-9a5b-0cf8918be895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:41.333806 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59be62-a35e-4de1-9a5b-0cf8918be895 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:41.334509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59be62-a35e-4de1-9a5b-0cf8918be895 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:41.335114 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59be62-a35e-4de1-9a5b-0cf8918be895 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:41.336398 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c59be62-a35e-4de1-9a5b-0cf8918be895 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:41.340284 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c59be62-a35e-4de1-9a5b-0cf8918be895 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:41.406877 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1244/2499] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:00:41 2022] POST /identity/v3/projects => generated 387 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:00:41.410465 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58d976-3133-48f3-9a29-57970d3c1ebc None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:41.423534 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58d976-3133-48f3-9a29-57970d3c1ebc None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:41.522968 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3feb9f-bd1f-4dc3-8c1c-c7cfd71c7771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:41.524330 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3feb9f-bd1f-4dc3-8c1c-c7cfd71c7771 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:41.524858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3feb9f-bd1f-4dc3-8c1c-c7cfd71c7771 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:41.525138 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3feb9f-bd1f-4dc3-8c1c-c7cfd71c7771 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:41.525862 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3feb9f-bd1f-4dc3-8c1c-c7cfd71c7771 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:41.530658 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3feb9f-bd1f-4dc3-8c1c-c7cfd71c7771 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:41.592662 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1256/2500] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:00:41 2022] POST /identity/v3/users => generated 300 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 09:00:41.596548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c59be62-a35e-4de1-9a5b-0cf8918be895 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:41.611792 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c59be62-a35e-4de1-9a5b-0cf8918be895 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:41.625972 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f121c-4426-4e6a-b086-d252a1b659a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:41.626916 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f121c-4426-4e6a-b086-d252a1b659a6 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:41.628200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f121c-4426-4e6a-b086-d252a1b659a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:41.628200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f121c-4426-4e6a-b086-d252a1b659a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:41.629280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f121c-4426-4e6a-b086-d252a1b659a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:41.632972 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f121c-4426-4e6a-b086-d252a1b659a6 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:41.664633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1245/2501] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:00:41.670274 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3feb9f-bd1f-4dc3-8c1c-c7cfd71c7771 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:41.686080 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3feb9f-bd1f-4dc3-8c1c-c7cfd71c7771 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:41.693061 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a143d0-f484-456f-8dae-4dbaea0eab76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:41.693932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a143d0-f484-456f-8dae-4dbaea0eab76 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:41.694258 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a143d0-f484-456f-8dae-4dbaea0eab76 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:41.694627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a143d0-f484-456f-8dae-4dbaea0eab76 admin admin] PATH_INFO: `/v3/projects/d4acf64c333742fab9e2384db37ebb07/users/9f9ba1ff1a4043eba4ae50198931147d/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:41.742457 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a143d0-f484-456f-8dae-4dbaea0eab76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4acf64c333742fab9e2384db37ebb07, user_id=9f9ba1ff1a4043eba4ae50198931147d, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:41.746180 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a143d0-f484-456f-8dae-4dbaea0eab76 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:41.766687 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:41.787521 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1257/2502] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:00:41 2022] PUT /identity/v3/projects/d4acf64c333742fab9e2384db37ebb07/users/9f9ba1ff1a4043eba4ae50198931147d/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:41.792679 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f121c-4426-4e6a-b086-d252a1b659a6 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:41.816891 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f121c-4426-4e6a-b086-d252a1b659a6 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:41.932830 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c3a76-573b-4190-bb9a-bc5ed8271a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:41.933474 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c3a76-573b-4190-bb9a-bc5ed8271a43 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:41.933754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c3a76-573b-4190-bb9a-bc5ed8271a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:41.934031 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c3a76-573b-4190-bb9a-bc5ed8271a43 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:41.934686 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c3a76-573b-4190-bb9a-bc5ed8271a43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:41.936413 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c3a76-573b-4190-bb9a-bc5ed8271a43 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:41.949028 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1246/2503] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:00:41.953690 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a143d0-f484-456f-8dae-4dbaea0eab76 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:41.966058 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a143d0-f484-456f-8dae-4dbaea0eab76 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:41.971106 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e92ecb-fc60-4bcb-b367-4616ea2e086e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:41.972436 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e92ecb-fc60-4bcb-b367-4616ea2e086e admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:41.972754 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e92ecb-fc60-4bcb-b367-4616ea2e086e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:41.973060 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e92ecb-fc60-4bcb-b367-4616ea2e086e admin admin] PATH_INFO: `/v3/projects/d4acf64c333742fab9e2384db37ebb07/users/9f9ba1ff1a4043eba4ae50198931147d/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:41.975212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e92ecb-fc60-4bcb-b367-4616ea2e086e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4acf64c333742fab9e2384db37ebb07, user_id=9f9ba1ff1a4043eba4ae50198931147d, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:41.977892 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e92ecb-fc60-4bcb-b367-4616ea2e086e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:41.995601 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1258/2504] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:00:41 2022] PUT /identity/v3/projects/d4acf64c333742fab9e2384db37ebb07/users/9f9ba1ff1a4043eba4ae50198931147d/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:42.007507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3510954-23a6-4af8-8bd1-2578829e28da None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:42.007507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3510954-23a6-4af8-8bd1-2578829e28da None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:42.007859 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3510954-23a6-4af8-8bd1-2578829e28da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:42.054594 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b3510954-23a6-4af8-8bd1-2578829e28da None None] MFA Rules not processed for user `9f9ba1ff1a4043eba4ae50198931147d`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:42.057005 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b3510954-23a6-4af8-8bd1-2578829e28da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:42.063897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e92ecb-fc60-4bcb-b367-4616ea2e086e admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:42.066506 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c5e92ecb-fc60-4bcb-b367-4616ea2e086e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:42.129637 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:42.178470 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1247/2505] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:42 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:42.184524 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e92ecb-fc60-4bcb-b367-4616ea2e086e admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:42.191246 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b035fa33-3ed5-4e96-99ea-5854d09d85e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:42.192149 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035fa33-3ed5-4e96-99ea-5854d09d85e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:42.192720 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035fa33-3ed5-4e96-99ea-5854d09d85e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:42.193261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035fa33-3ed5-4e96-99ea-5854d09d85e1 admin admin] PATH_INFO: `/v3/users/229fe8f73e074ef89b1110da844268df` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:42.195298 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b035fa33-3ed5-4e96-99ea-5854d09d85e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=229fe8f73e074ef89b1110da844268df)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:42.195675 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3510954-23a6-4af8-8bd1-2578829e28da None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:42.197309 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b3510954-23a6-4af8-8bd1-2578829e28da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:42.198217 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b035fa33-3ed5-4e96-99ea-5854d09d85e1 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:42.283522 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b035fa33-3ed5-4e96-99ea-5854d09d85e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '229fe8f73e074ef89b1110da844268df'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:42.290514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3510954-23a6-4af8-8bd1-2578829e28da None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:42.298252 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf554e-1939-4bbd-af8e-60e00c32e99c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:42.298993 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf554e-1939-4bbd-af8e-60e00c32e99c service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:42.299519 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf554e-1939-4bbd-af8e-60e00c32e99c service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:42.299958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebf554e-1939-4bbd-af8e-60e00c32e99c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:42.305793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b035fa33-3ed5-4e96-99ea-5854d09d85e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '229fe8f73e074ef89b1110da844268df'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:42.315949 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf554e-1939-4bbd-af8e-60e00c32e99c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:42.318522 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebf554e-1939-4bbd-af8e-60e00c32e99c service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:42.323127 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b035fa33-3ed5-4e96-99ea-5854d09d85e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '229fe8f73e074ef89b1110da844268df'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:42.335710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1248/2506] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:00:42 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:00:42.342456 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b035fa33-3ed5-4e96-99ea-5854d09d85e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '229fe8f73e074ef89b1110da844268df'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:42.343261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b035fa33-3ed5-4e96-99ea-5854d09d85e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '229fe8f73e074ef89b1110da844268df'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:42.352203 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1259/2507] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:00:42 2022] DELETE /identity/v3/users/229fe8f73e074ef89b1110da844268df => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:42.618328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf554e-1939-4bbd-af8e-60e00c32e99c service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:42.633828 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebf554e-1939-4bbd-af8e-60e00c32e99c service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:42.722712 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebcdfa0-6668-4ded-9525-e7f7df8c7031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:42.723448 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebcdfa0-6668-4ded-9525-e7f7df8c7031 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:42.723752 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebcdfa0-6668-4ded-9525-e7f7df8c7031 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:42.724170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebcdfa0-6668-4ded-9525-e7f7df8c7031 admin admin] PATH_INFO: `/v3/projects/37cc63bca9274f9ba58f8a92a499a36d` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:42.725522 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebcdfa0-6668-4ded-9525-e7f7df8c7031 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37cc63bca9274f9ba58f8a92a499a36d)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:42.728577 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebcdfa0-6668-4ded-9525-e7f7df8c7031 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:42.805312 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-2ebcdfa0-6668-4ded-9525-e7f7df8c7031 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37cc63bca9274f9ba58f8a92a499a36d', 'request_id': 'req-2ebcdfa0-6668-4ded-9525-e7f7df8c7031'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:42.819794 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-2ebcdfa0-6668-4ded-9525-e7f7df8c7031 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37cc63bca9274f9ba58f8a92a499a36d', 'request_id': 'req-2ebcdfa0-6668-4ded-9525-e7f7df8c7031'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:42.828458 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-2ebcdfa0-6668-4ded-9525-e7f7df8c7031 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37cc63bca9274f9ba58f8a92a499a36d', 'request_id': 'req-2ebcdfa0-6668-4ded-9525-e7f7df8c7031'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:42.847241 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-2ebcdfa0-6668-4ded-9525-e7f7df8c7031 admin admin] The token cache is being invalidate because project 37cc63bca9274f9ba58f8a92a499a36d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:00:42.847578 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-2ebcdfa0-6668-4ded-9525-e7f7df8c7031 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:42.848550 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-2ebcdfa0-6668-4ded-9525-e7f7df8c7031 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:42.850874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1249/2508] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:00:42 2022] DELETE /identity/v3/projects/37cc63bca9274f9ba58f8a92a499a36d => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:42.856418 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b035fa33-3ed5-4e96-99ea-5854d09d85e1 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:42.858939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b035fa33-3ed5-4e96-99ea-5854d09d85e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:42.881352 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b035fa33-3ed5-4e96-99ea-5854d09d85e1 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:42.982676 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fba29e-12bf-43a2-aa1f-485667718e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:42.983509 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fba29e-12bf-43a2-aa1f-485667718e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:42.983959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fba29e-12bf-43a2-aa1f-485667718e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:42.984427 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fba29e-12bf-43a2-aa1f-485667718e06 admin admin] PATH_INFO: `/v3/users/d4460301d0e94eeca47cd87e4d04299a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:42.986723 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fba29e-12bf-43a2-aa1f-485667718e06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4460301d0e94eeca47cd87e4d04299a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:42.990238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fba29e-12bf-43a2-aa1f-485667718e06 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:42.998411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:43.077353 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-38fba29e-12bf-43a2-aa1f-485667718e06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4460301d0e94eeca47cd87e4d04299a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:43.103655 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-38fba29e-12bf-43a2-aa1f-485667718e06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4460301d0e94eeca47cd87e4d04299a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:43.115343 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-38fba29e-12bf-43a2-aa1f-485667718e06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4460301d0e94eeca47cd87e4d04299a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:43.126102 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-38fba29e-12bf-43a2-aa1f-485667718e06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4460301d0e94eeca47cd87e4d04299a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:43.127084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-38fba29e-12bf-43a2-aa1f-485667718e06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4460301d0e94eeca47cd87e4d04299a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:43.135194 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1260/2509] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:00:42 2022] DELETE /identity/v3/users/d4460301d0e94eeca47cd87e4d04299a => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:43.185045 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebcdfa0-6668-4ded-9525-e7f7df8c7031 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:43.187949 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-2ebcdfa0-6668-4ded-9525-e7f7df8c7031 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:43.201469 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:43.295307 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebcdfa0-6668-4ded-9525-e7f7df8c7031 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:43.387628 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51410d1-2dd8-4d77-a5b2-feafea24a0f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:43.388213 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51410d1-2dd8-4d77-a5b2-feafea24a0f7 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:43.388616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51410d1-2dd8-4d77-a5b2-feafea24a0f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:43.388904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51410d1-2dd8-4d77-a5b2-feafea24a0f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:43.391974 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d51410d1-2dd8-4d77-a5b2-feafea24a0f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:43.442618 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fba29e-12bf-43a2-aa1f-485667718e06 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:43.444247 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-38fba29e-12bf-43a2-aa1f-485667718e06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:43.520488 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51410d1-2dd8-4d77-a5b2-feafea24a0f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:43.522528 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51410d1-2dd8-4d77-a5b2-feafea24a0f7 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:43.537461 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1250/2510] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:00:43 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:00:43.563054 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fba29e-12bf-43a2-aa1f-485667718e06 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:43.573740 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4917071-500a-4433-a102-26fad5e228d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:43.574477 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4917071-500a-4433-a102-26fad5e228d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:43.574754 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4917071-500a-4433-a102-26fad5e228d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:43.575139 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4917071-500a-4433-a102-26fad5e228d6 admin admin] PATH_INFO: `/v3/projects/37a83a062eee4e989da032ca1857a68d` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:43.576249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4917071-500a-4433-a102-26fad5e228d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37a83a062eee4e989da032ca1857a68d)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:43.578193 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4917071-500a-4433-a102-26fad5e228d6 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:43.661770 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c4917071-500a-4433-a102-26fad5e228d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37a83a062eee4e989da032ca1857a68d', 'request_id': 'req-c4917071-500a-4433-a102-26fad5e228d6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:43.672178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c4917071-500a-4433-a102-26fad5e228d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37a83a062eee4e989da032ca1857a68d', 'request_id': 'req-c4917071-500a-4433-a102-26fad5e228d6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:43.685503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c4917071-500a-4433-a102-26fad5e228d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37a83a062eee4e989da032ca1857a68d', 'request_id': 'req-c4917071-500a-4433-a102-26fad5e228d6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:43.713212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c4917071-500a-4433-a102-26fad5e228d6 admin admin] The token cache is being invalidate because project 37a83a062eee4e989da032ca1857a68d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:00:43.714137 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c4917071-500a-4433-a102-26fad5e228d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:43.715212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c4917071-500a-4433-a102-26fad5e228d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:43.716996 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1261/2511] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:00:43 2022] DELETE /identity/v3/projects/37a83a062eee4e989da032ca1857a68d => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 09:00:43.727474 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b58f8a-77e2-45a3-9337-11ef45fac9ee None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:43.728127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b58f8a-77e2-45a3-9337-11ef45fac9ee None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:43.728127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b58f8a-77e2-45a3-9337-11ef45fac9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:43.759177 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-66b58f8a-77e2-45a3-9337-11ef45fac9ee None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:43.761424 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-66b58f8a-77e2-45a3-9337-11ef45fac9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:43.871917 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1251/2512] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:43.885995 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e8cc0-2226-44a8-8afd-53ed09c407eb None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:43.886739 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e8cc0-2226-44a8-8afd-53ed09c407eb None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:43.886739 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e8cc0-2226-44a8-8afd-53ed09c407eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:43.915115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-213e8cc0-2226-44a8-8afd-53ed09c407eb None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:43.916943 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-213e8cc0-2226-44a8-8afd-53ed09c407eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:43.929771 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1262/2513] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:43.941599 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b58f8a-77e2-45a3-9337-11ef45fac9ee None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:43.965545 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b58f8a-77e2-45a3-9337-11ef45fac9ee None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:43.973454 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9d3c4-0d84-4f09-a5ec-8629195e2927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:43.974263 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9d3c4-0d84-4f09-a5ec-8629195e2927 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:43.974867 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9d3c4-0d84-4f09-a5ec-8629195e2927 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:43.975334 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9d3c4-0d84-4f09-a5ec-8629195e2927 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:43.976163 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c9d3c4-0d84-4f09-a5ec-8629195e2927 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 09:00:43.976730 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c9d3c4-0d84-4f09-a5ec-8629195e2927 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:43.978644 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c9d3c4-0d84-4f09-a5ec-8629195e2927 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:43.999109 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1252/2514] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 09:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 09:00:44.006378 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e8cc0-2226-44a8-8afd-53ed09c407eb None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:44.024385 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e8cc0-2226-44a8-8afd-53ed09c407eb None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:44.032381 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b70fdc-b88b-4f5b-88cb-4258f2800229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:44.033551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b70fdc-b88b-4f5b-88cb-4258f2800229 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:44.033896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b70fdc-b88b-4f5b-88cb-4258f2800229 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:44.034353 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b70fdc-b88b-4f5b-88cb-4258f2800229 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:44.035445 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b70fdc-b88b-4f5b-88cb-4258f2800229 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:44.039694 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b70fdc-b88b-4f5b-88cb-4258f2800229 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:44.051896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:44.093753 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1263/2515] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:00:44 2022] POST /identity/v3/projects => generated 379 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:00:44.098937 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9d3c4-0d84-4f09-a5ec-8629195e2927 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:44.110159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:44.118961 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9d3c4-0d84-4f09-a5ec-8629195e2927 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:44.213381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8694b502-d251-4a4e-b595-859f6ec1a0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:44.214007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694b502-d251-4a4e-b595-859f6ec1a0a8 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:44.214310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694b502-d251-4a4e-b595-859f6ec1a0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:44.214559 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694b502-d251-4a4e-b595-859f6ec1a0a8 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:44.215336 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8694b502-d251-4a4e-b595-859f6ec1a0a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:44.217234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8694b502-d251-4a4e-b595-859f6ec1a0a8 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:44.243172 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1253/2516] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:00:44 2022] POST /identity/v3/users => generated 290 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:00:44.248052 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b70fdc-b88b-4f5b-88cb-4258f2800229 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:44.260081 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b70fdc-b88b-4f5b-88cb-4258f2800229 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:44.265403 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db399128-dac2-419a-832c-891f215d1dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:44.265970 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db399128-dac2-419a-832c-891f215d1dfe admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:44.266257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db399128-dac2-419a-832c-891f215d1dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:44.266552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db399128-dac2-419a-832c-891f215d1dfe admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:44.267251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db399128-dac2-419a-832c-891f215d1dfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:44.268967 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db399128-dac2-419a-832c-891f215d1dfe admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:44.291122 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1264/2517] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 09:00:44.295917 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8694b502-d251-4a4e-b595-859f6ec1a0a8 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:44.314712 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8694b502-d251-4a4e-b595-859f6ec1a0a8 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:44.324029 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c26d8de-d435-41a2-881d-a3e9d4a7b9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:44.325167 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26d8de-d435-41a2-881d-a3e9d4a7b9f6 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:44.325713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26d8de-d435-41a2-881d-a3e9d4a7b9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:44.326178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26d8de-d435-41a2-881d-a3e9d4a7b9f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:44.327373 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c26d8de-d435-41a2-881d-a3e9d4a7b9f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:44.330669 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c26d8de-d435-41a2-881d-a3e9d4a7b9f6 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:44.352724 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1254/2518] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:00:44.357952 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db399128-dac2-419a-832c-891f215d1dfe admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:44.377112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db399128-dac2-419a-832c-891f215d1dfe admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:44.386423 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd65630-5950-4ac5-9db5-06adebb13c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:44.386816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd65630-5950-4ac5-9db5-06adebb13c6e admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:44.387424 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd65630-5950-4ac5-9db5-06adebb13c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:44.387656 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd65630-5950-4ac5-9db5-06adebb13c6e admin admin] PATH_INFO: `/v3/projects/6773ba16da8845d39c90a627802cf05f/users/34eaab912a91496daeedbaf63f5ddcbb/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:44.408381 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd65630-5950-4ac5-9db5-06adebb13c6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6773ba16da8845d39c90a627802cf05f, user_id=34eaab912a91496daeedbaf63f5ddcbb, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:44.411713 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd65630-5950-4ac5-9db5-06adebb13c6e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:44.434375 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1265/2519] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:00:44 2022] PUT /identity/v3/projects/6773ba16da8845d39c90a627802cf05f/users/34eaab912a91496daeedbaf63f5ddcbb/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:44.441300 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c223f8-e902-450f-ac5a-55f1a106a706 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:44.441614 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c223f8-e902-450f-ac5a-55f1a106a706 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:44.441891 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c223f8-e902-450f-ac5a-55f1a106a706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:44.465965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b5c223f8-e902-450f-ac5a-55f1a106a706 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:44.467363 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b5c223f8-e902-450f-ac5a-55f1a106a706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:44.539851 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1255/2520] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:44.550332 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd65630-5950-4ac5-9db5-06adebb13c6e admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:44.552221 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ecd65630-5950-4ac5-9db5-06adebb13c6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:44.614903 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd65630-5950-4ac5-9db5-06adebb13c6e admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:44.618773 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268dc299-c339-4ff2-9811-93914c47f780 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:44.619435 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268dc299-c339-4ff2-9811-93914c47f780 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:44.619708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268dc299-c339-4ff2-9811-93914c47f780 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:44.619969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268dc299-c339-4ff2-9811-93914c47f780 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:44.632033 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268dc299-c339-4ff2-9811-93914c47f780 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:44.633503 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268dc299-c339-4ff2-9811-93914c47f780 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:44.644964 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1266/2521] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:00:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:00:48.290525 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c223f8-e902-450f-ac5a-55f1a106a706 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:48.293190 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b5c223f8-e902-450f-ac5a-55f1a106a706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:48.413542 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c223f8-e902-450f-ac5a-55f1a106a706 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:48.421570 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa297e-28eb-4bef-9f28-1b3dca1569e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:48.423377 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa297e-28eb-4bef-9f28-1b3dca1569e5 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:48.424699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa297e-28eb-4bef-9f28-1b3dca1569e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:48.425139 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa297e-28eb-4bef-9f28-1b3dca1569e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:48.431032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1afa297e-28eb-4bef-9f28-1b3dca1569e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:48.498000 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:48.533283 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afa297e-28eb-4bef-9f28-1b3dca1569e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:48.537853 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afa297e-28eb-4bef-9f28-1b3dca1569e5 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:48.552892 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1256/2522] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:00:48 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:00:50.075122 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c6dbf-a6c2-42f2-9059-af68756565c9 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:50.075731 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c6dbf-a6c2-42f2-9059-af68756565c9 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:50.076123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c6dbf-a6c2-42f2-9059-af68756565c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:50.118814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-469c6dbf-a6c2-42f2-9059-af68756565c9 None None] MFA Rules not processed for user `34eaab912a91496daeedbaf63f5ddcbb`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:50.120114 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-469c6dbf-a6c2-42f2-9059-af68756565c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:50.225841 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1267/2523] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:50.231099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa297e-28eb-4bef-9f28-1b3dca1569e5 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:50.245129 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa297e-28eb-4bef-9f28-1b3dca1569e5 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:50.250395 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb33257-7e17-494a-9a66-14778c4d0787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:50.251300 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb33257-7e17-494a-9a66-14778c4d0787 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:50.251728 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb33257-7e17-494a-9a66-14778c4d0787 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:50.252091 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb33257-7e17-494a-9a66-14778c4d0787 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:50.252754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb33257-7e17-494a-9a66-14778c4d0787 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:50.255087 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb33257-7e17-494a-9a66-14778c4d0787 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:50.291523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1257/2524] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:00:50 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:00:50.296042 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c6dbf-a6c2-42f2-9059-af68756565c9 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:50.312989 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c6dbf-a6c2-42f2-9059-af68756565c9 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:50.369463 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:50.398440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a309e9b-9820-4589-bd1c-3782d31cdd24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:50.399100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a309e9b-9820-4589-bd1c-3782d31cdd24 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:50.399357 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a309e9b-9820-4589-bd1c-3782d31cdd24 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:50.399650 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a309e9b-9820-4589-bd1c-3782d31cdd24 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:50.400292 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a309e9b-9820-4589-bd1c-3782d31cdd24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:50.402240 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a309e9b-9820-4589-bd1c-3782d31cdd24 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:50.426453 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1268/2525] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:00:50 2022] POST /identity/v3/users => generated 295 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:00:50.431465 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb33257-7e17-494a-9a66-14778c4d0787 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:50.448675 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb33257-7e17-494a-9a66-14778c4d0787 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:50.461046 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c5a09-67d0-463c-9af9-c30d4c7835c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:50.461698 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c5a09-67d0-463c-9af9-c30d4c7835c6 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:50.461973 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c5a09-67d0-463c-9af9-c30d4c7835c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:50.462246 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c5a09-67d0-463c-9af9-c30d4c7835c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:50.463046 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c5a09-67d0-463c-9af9-c30d4c7835c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:50.465024 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c5a09-67d0-463c-9af9-c30d4c7835c6 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:50.479089 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1258/2526] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:00:50.483886 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a309e9b-9820-4589-bd1c-3782d31cdd24 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:50.496337 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a309e9b-9820-4589-bd1c-3782d31cdd24 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:50.505233 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29bb0f-6c78-454f-b18f-f5b92e8d6a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:50.506309 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29bb0f-6c78-454f-b18f-f5b92e8d6a05 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:50.506878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29bb0f-6c78-454f-b18f-f5b92e8d6a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:50.507322 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29bb0f-6c78-454f-b18f-f5b92e8d6a05 admin admin] PATH_INFO: `/v3/projects/17ed1ac4468e42c5a8152cd5ce43eaad/users/b03c1f7daa07427888a86c7e8a69dd4d/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:50.529841 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29bb0f-6c78-454f-b18f-f5b92e8d6a05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17ed1ac4468e42c5a8152cd5ce43eaad, user_id=b03c1f7daa07427888a86c7e8a69dd4d, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:50.532565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29bb0f-6c78-454f-b18f-f5b92e8d6a05 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:50.553367 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1269/2527] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:00:50 2022] PUT /identity/v3/projects/17ed1ac4468e42c5a8152cd5ce43eaad/users/b03c1f7daa07427888a86c7e8a69dd4d/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:50.560495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c5a09-67d0-463c-9af9-c30d4c7835c6 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:50.580457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c5a09-67d0-463c-9af9-c30d4c7835c6 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:50.649501 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01f8f4c-70f1-4d6d-8a64-87efcd12599a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:50.650133 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f8f4c-70f1-4d6d-8a64-87efcd12599a admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:50.650406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f8f4c-70f1-4d6d-8a64-87efcd12599a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:50.650679 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f8f4c-70f1-4d6d-8a64-87efcd12599a admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:50.651390 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01f8f4c-70f1-4d6d-8a64-87efcd12599a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:50.652999 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01f8f4c-70f1-4d6d-8a64-87efcd12599a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:50.664112 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1259/2528] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:00:50.679512 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29bb0f-6c78-454f-b18f-f5b92e8d6a05 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:50.690104 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29bb0f-6c78-454f-b18f-f5b92e8d6a05 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:50.701155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a5227-f1cb-4ea3-8866-338a7bc4fac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:50.702718 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a5227-f1cb-4ea3-8866-338a7bc4fac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:50.702980 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a5227-f1cb-4ea3-8866-338a7bc4fac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:50.703796 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a5227-f1cb-4ea3-8866-338a7bc4fac7 admin admin] PATH_INFO: `/v3/projects/17ed1ac4468e42c5a8152cd5ce43eaad/users/b03c1f7daa07427888a86c7e8a69dd4d/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:50.709701 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a5227-f1cb-4ea3-8866-338a7bc4fac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17ed1ac4468e42c5a8152cd5ce43eaad, user_id=b03c1f7daa07427888a86c7e8a69dd4d, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:50.714702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a5227-f1cb-4ea3-8866-338a7bc4fac7 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:50.729801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1270/2529] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:00:50 2022] PUT /identity/v3/projects/17ed1ac4468e42c5a8152cd5ce43eaad/users/b03c1f7daa07427888a86c7e8a69dd4d/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:51.545589 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01f8f4c-70f1-4d6d-8a64-87efcd12599a admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:51.547587 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d01f8f4c-70f1-4d6d-8a64-87efcd12599a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:51.564093 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01f8f4c-70f1-4d6d-8a64-87efcd12599a admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:51.658704 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72f1eb-8e9f-4b3b-95cc-740a1ca246b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:51.659658 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72f1eb-8e9f-4b3b-95cc-740a1ca246b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:51.660112 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72f1eb-8e9f-4b3b-95cc-740a1ca246b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:51.660552 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72f1eb-8e9f-4b3b-95cc-740a1ca246b3 admin admin] PATH_INFO: `/v3/users/2a9ee9b775404234b20724380569382c` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:51.661902 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72f1eb-8e9f-4b3b-95cc-740a1ca246b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a9ee9b775404234b20724380569382c)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:51.664251 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72f1eb-8e9f-4b3b-95cc-740a1ca246b3 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:51.760316 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:51.764343 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5b72f1eb-8e9f-4b3b-95cc-740a1ca246b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a9ee9b775404234b20724380569382c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:51.783695 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5b72f1eb-8e9f-4b3b-95cc-740a1ca246b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a9ee9b775404234b20724380569382c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:51.792033 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5b72f1eb-8e9f-4b3b-95cc-740a1ca246b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a9ee9b775404234b20724380569382c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:51.793401 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5b72f1eb-8e9f-4b3b-95cc-740a1ca246b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a9ee9b775404234b20724380569382c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:51.794365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-5b72f1eb-8e9f-4b3b-95cc-740a1ca246b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a9ee9b775404234b20724380569382c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:51.822578 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1260/2530] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:00:51 2022] DELETE /identity/v3/users/2a9ee9b775404234b20724380569382c => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:51.826312 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbab39a-c3b0-444b-b190-335d206122ea None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:51.827791 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbab39a-c3b0-444b-b190-335d206122ea None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:51.828364 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbab39a-c3b0-444b-b190-335d206122ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:51.851376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-3dbab39a-c3b0-444b-b190-335d206122ea None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:51.853016 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3dbab39a-c3b0-444b-b190-335d206122ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:51.942258 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1271/2531] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:51.966577 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72f1eb-8e9f-4b3b-95cc-740a1ca246b3 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:51.970870 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-5b72f1eb-8e9f-4b3b-95cc-740a1ca246b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:52.051967 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72f1eb-8e9f-4b3b-95cc-740a1ca246b3 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:52.056070 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e8ed6-a720-4635-bf50-dfe9efb84b9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:52.056783 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e8ed6-a720-4635-bf50-dfe9efb84b9c service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:52.057131 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e8ed6-a720-4635-bf50-dfe9efb84b9c service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:52.057474 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e8ed6-a720-4635-bf50-dfe9efb84b9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:52.069876 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e8ed6-a720-4635-bf50-dfe9efb84b9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:52.071709 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e8ed6-a720-4635-bf50-dfe9efb84b9c service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:52.086870 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1261/2532] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:00:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 09:00:52.619922 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbab39a-c3b0-444b-b190-335d206122ea None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:52.621593 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3dbab39a-c3b0-444b-b190-335d206122ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:52.649086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbab39a-c3b0-444b-b190-335d206122ea None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:52.663172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f570518-8997-40fc-a4c0-debae3de9e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:52.663989 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f570518-8997-40fc-a4c0-debae3de9e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:52.664362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f570518-8997-40fc-a4c0-debae3de9e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:52.664742 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f570518-8997-40fc-a4c0-debae3de9e70 admin admin] PATH_INFO: `/v3/projects/eaf91ca61a914adeab3c2450b8b6191d` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:52.666338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f570518-8997-40fc-a4c0-debae3de9e70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaf91ca61a914adeab3c2450b8b6191d)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:52.668572 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f570518-8997-40fc-a4c0-debae3de9e70 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:52.786235 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8f570518-8997-40fc-a4c0-debae3de9e70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaf91ca61a914adeab3c2450b8b6191d', 'request_id': 'req-8f570518-8997-40fc-a4c0-debae3de9e70'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:52.795272 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8f570518-8997-40fc-a4c0-debae3de9e70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaf91ca61a914adeab3c2450b8b6191d', 'request_id': 'req-8f570518-8997-40fc-a4c0-debae3de9e70'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:52.809897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8f570518-8997-40fc-a4c0-debae3de9e70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaf91ca61a914adeab3c2450b8b6191d', 'request_id': 'req-8f570518-8997-40fc-a4c0-debae3de9e70'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:52.834085 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8f570518-8997-40fc-a4c0-debae3de9e70 admin admin] The token cache is being invalidate because project eaf91ca61a914adeab3c2450b8b6191d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:00:52.834715 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8f570518-8997-40fc-a4c0-debae3de9e70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:52.835996 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8f570518-8997-40fc-a4c0-debae3de9e70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:52.839647 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1272/2533] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:00:52 2022] DELETE /identity/v3/projects/eaf91ca61a914adeab3c2450b8b6191d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:52.844505 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e8ed6-a720-4635-bf50-dfe9efb84b9c service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:52.861788 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e8ed6-a720-4635-bf50-dfe9efb84b9c service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:52.948960 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e6411-5a7e-48ee-b54e-e310d5b02572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:52.949620 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e6411-5a7e-48ee-b54e-e310d5b02572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:52.950479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e6411-5a7e-48ee-b54e-e310d5b02572 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:52.951180 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e6411-5a7e-48ee-b54e-e310d5b02572 admin admin] PATH_INFO: `/v3/users/cbfce778eb7f43da88958b552039dd63` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:52.954767 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e6411-5a7e-48ee-b54e-e310d5b02572 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbfce778eb7f43da88958b552039dd63)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:52.956988 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e6411-5a7e-48ee-b54e-e310d5b02572 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:53.046365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-be5e6411-5a7e-48ee-b54e-e310d5b02572 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbfce778eb7f43da88958b552039dd63'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:53.060497 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-be5e6411-5a7e-48ee-b54e-e310d5b02572 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbfce778eb7f43da88958b552039dd63'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:53.072144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-be5e6411-5a7e-48ee-b54e-e310d5b02572 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbfce778eb7f43da88958b552039dd63'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:53.073242 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-be5e6411-5a7e-48ee-b54e-e310d5b02572 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbfce778eb7f43da88958b552039dd63'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:53.075294 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-be5e6411-5a7e-48ee-b54e-e310d5b02572 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbfce778eb7f43da88958b552039dd63'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:53.078710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:53.105776 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1262/2534] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:00:52 2022] DELETE /identity/v3/users/cbfce778eb7f43da88958b552039dd63 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:53.976990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f570518-8997-40fc-a4c0-debae3de9e70 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:53.987479 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8f570518-8997-40fc-a4c0-debae3de9e70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:54.017057 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f570518-8997-40fc-a4c0-debae3de9e70 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:54.074133 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:54.159368 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d2126-36b6-4063-b9b3-f93d6ce07ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:54.160624 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d2126-36b6-4063-b9b3-f93d6ce07ce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:54.161184 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d2126-36b6-4063-b9b3-f93d6ce07ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:54.161576 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d2126-36b6-4063-b9b3-f93d6ce07ce6 admin admin] PATH_INFO: `/v3/projects/49384cd8c94f4098b709b38ed2d2d9f7` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:54.163067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d2126-36b6-4063-b9b3-f93d6ce07ce6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49384cd8c94f4098b709b38ed2d2d9f7)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:54.165951 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d2126-36b6-4063-b9b3-f93d6ce07ce6 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:54.248418 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-578d2126-36b6-4063-b9b3-f93d6ce07ce6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49384cd8c94f4098b709b38ed2d2d9f7', 'request_id': 'req-578d2126-36b6-4063-b9b3-f93d6ce07ce6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:54.264221 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-578d2126-36b6-4063-b9b3-f93d6ce07ce6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49384cd8c94f4098b709b38ed2d2d9f7', 'request_id': 'req-578d2126-36b6-4063-b9b3-f93d6ce07ce6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:54.278672 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-578d2126-36b6-4063-b9b3-f93d6ce07ce6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49384cd8c94f4098b709b38ed2d2d9f7', 'request_id': 'req-578d2126-36b6-4063-b9b3-f93d6ce07ce6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:54.296099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-578d2126-36b6-4063-b9b3-f93d6ce07ce6 admin admin] The token cache is being invalidate because project 49384cd8c94f4098b709b38ed2d2d9f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:00:54.296378 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-578d2126-36b6-4063-b9b3-f93d6ce07ce6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:54.297105 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-578d2126-36b6-4063-b9b3-f93d6ce07ce6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:54.299112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1273/2535] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:00:53 2022] DELETE /identity/v3/projects/49384cd8c94f4098b709b38ed2d2d9f7 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:54.303237 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e6411-5a7e-48ee-b54e-e310d5b02572 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:54.305388 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-be5e6411-5a7e-48ee-b54e-e310d5b02572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:54.322114 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e6411-5a7e-48ee-b54e-e310d5b02572 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:54.460183 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0bf69d-f7fa-4040-baf7-48b11265e34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:54.460979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bf69d-f7fa-4040-baf7-48b11265e34d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:54.461327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bf69d-f7fa-4040-baf7-48b11265e34d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:54.461670 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bf69d-f7fa-4040-baf7-48b11265e34d admin admin] PATH_INFO: `/v3/users/9f9ba1ff1a4043eba4ae50198931147d` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:54.463092 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0bf69d-f7fa-4040-baf7-48b11265e34d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f9ba1ff1a4043eba4ae50198931147d)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:54.466056 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0bf69d-f7fa-4040-baf7-48b11265e34d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:54.595557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0e0bf69d-f7fa-4040-baf7-48b11265e34d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f9ba1ff1a4043eba4ae50198931147d'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:54.618466 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0e0bf69d-f7fa-4040-baf7-48b11265e34d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f9ba1ff1a4043eba4ae50198931147d'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:54.645349 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0e0bf69d-f7fa-4040-baf7-48b11265e34d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f9ba1ff1a4043eba4ae50198931147d'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:54.651548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0e0bf69d-f7fa-4040-baf7-48b11265e34d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f9ba1ff1a4043eba4ae50198931147d'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:54.652259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-0e0bf69d-f7fa-4040-baf7-48b11265e34d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f9ba1ff1a4043eba4ae50198931147d'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:54.750021 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1263/2536] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:00:54 2022] DELETE /identity/v3/users/9f9ba1ff1a4043eba4ae50198931147d => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:55.818421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d2126-36b6-4063-b9b3-f93d6ce07ce6 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:55.821214 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-578d2126-36b6-4063-b9b3-f93d6ce07ce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:55.849807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d2126-36b6-4063-b9b3-f93d6ce07ce6 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:56.029856 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2779a-ad95-44a5-8b4d-e6ad31c6ab91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:56.031447 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2779a-ad95-44a5-8b4d-e6ad31c6ab91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:56.032291 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2779a-ad95-44a5-8b4d-e6ad31c6ab91 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:56.033050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2779a-ad95-44a5-8b4d-e6ad31c6ab91 admin admin] PATH_INFO: `/v3/projects/d4acf64c333742fab9e2384db37ebb07` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:56.036683 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc2779a-ad95-44a5-8b4d-e6ad31c6ab91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4acf64c333742fab9e2384db37ebb07)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:56.041334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc2779a-ad95-44a5-8b4d-e6ad31c6ab91 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:56.193742 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cbc2779a-ad95-44a5-8b4d-e6ad31c6ab91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4acf64c333742fab9e2384db37ebb07', 'request_id': 'req-cbc2779a-ad95-44a5-8b4d-e6ad31c6ab91'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:56.198665 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:56.215538 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cbc2779a-ad95-44a5-8b4d-e6ad31c6ab91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4acf64c333742fab9e2384db37ebb07', 'request_id': 'req-cbc2779a-ad95-44a5-8b4d-e6ad31c6ab91'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:56.237884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cbc2779a-ad95-44a5-8b4d-e6ad31c6ab91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4acf64c333742fab9e2384db37ebb07', 'request_id': 'req-cbc2779a-ad95-44a5-8b4d-e6ad31c6ab91'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:56.276757 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cbc2779a-ad95-44a5-8b4d-e6ad31c6ab91 admin admin] The token cache is being invalidate because project d4acf64c333742fab9e2384db37ebb07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:00:56.277480 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cbc2779a-ad95-44a5-8b4d-e6ad31c6ab91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:56.278634 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-cbc2779a-ad95-44a5-8b4d-e6ad31c6ab91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:56.299637 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1274/2537] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:00:55 2022] DELETE /identity/v3/projects/d4acf64c333742fab9e2384db37ebb07 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:56.305081 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bc768-1ffa-4cbe-94e1-367cbb5beeef None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:56.305946 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bc768-1ffa-4cbe-94e1-367cbb5beeef None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:56.306385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bc768-1ffa-4cbe-94e1-367cbb5beeef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:56.363319 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-4a1bc768-1ffa-4cbe-94e1-367cbb5beeef None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:56.363319 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4a1bc768-1ffa-4cbe-94e1-367cbb5beeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:56.604562 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1264/2538] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:56.623459 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313f4f0-1dde-4a31-ba08-618139b20090 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:56.623971 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313f4f0-1dde-4a31-ba08-618139b20090 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:56.624426 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b313f4f0-1dde-4a31-ba08-618139b20090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:56.681342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-b313f4f0-1dde-4a31-ba08-618139b20090 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:56.690593 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b313f4f0-1dde-4a31-ba08-618139b20090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:56.732045 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1275/2539] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:56.735539 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1bc768-1ffa-4cbe-94e1-367cbb5beeef None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:56.748626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:56.772457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1bc768-1ffa-4cbe-94e1-367cbb5beeef None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:56.795156 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b5502-55ce-4340-a14b-35eef1ae594f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:56.799939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b5502-55ce-4340-a14b-35eef1ae594f admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:56.800611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b5502-55ce-4340-a14b-35eef1ae594f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:56.801112 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b5502-55ce-4340-a14b-35eef1ae594f admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:56.803666 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b5502-55ce-4340-a14b-35eef1ae594f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 09:00:56.804212 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b5502-55ce-4340-a14b-35eef1ae594f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:56.807444 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b5502-55ce-4340-a14b-35eef1ae594f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:56.873981 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1265/2540] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 09:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 09:00:56.880372 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b313f4f0-1dde-4a31-ba08-618139b20090 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:56.907582 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b313f4f0-1dde-4a31-ba08-618139b20090 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:56.938495 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484dcd6-174c-48a7-b2d2-a7db7505b999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:56.940140 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484dcd6-174c-48a7-b2d2-a7db7505b999 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:56.940872 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484dcd6-174c-48a7-b2d2-a7db7505b999 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:56.941570 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484dcd6-174c-48a7-b2d2-a7db7505b999 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:56.943240 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484dcd6-174c-48a7-b2d2-a7db7505b999 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:56.949943 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484dcd6-174c-48a7-b2d2-a7db7505b999 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:57.080269 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1276/2541] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:00:56 2022] POST /identity/v3/projects => generated 371 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:00:57.087039 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b5502-55ce-4340-a14b-35eef1ae594f admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:57.125089 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b5502-55ce-4340-a14b-35eef1ae594f admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:57.382811 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6ed76-645c-46c0-9e76-97c075214955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:57.383955 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6ed76-645c-46c0-9e76-97c075214955 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:57.384302 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6ed76-645c-46c0-9e76-97c075214955 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:57.384800 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d6ed76-645c-46c0-9e76-97c075214955 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:57.385900 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d6ed76-645c-46c0-9e76-97c075214955 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:57.390164 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d6ed76-645c-46c0-9e76-97c075214955 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:57.440799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1266/2542] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:00:57 2022] POST /identity/v3/users => generated 286 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:00:57.447717 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484dcd6-174c-48a7-b2d2-a7db7505b999 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:57.468080 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484dcd6-174c-48a7-b2d2-a7db7505b999 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:57.477462 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b09b37-75ae-477c-b634-ddb0c49c2459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:57.478983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b09b37-75ae-477c-b634-ddb0c49c2459 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:57.479596 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b09b37-75ae-477c-b634-ddb0c49c2459 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:57.480265 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b09b37-75ae-477c-b634-ddb0c49c2459 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:57.482236 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b09b37-75ae-477c-b634-ddb0c49c2459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:57.485853 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b09b37-75ae-477c-b634-ddb0c49c2459 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:57.518731 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6ed76-645c-46c0-9e76-97c075214955 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:57.529112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1277/2543] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:00:57.540261 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d6ed76-645c-46c0-9e76-97c075214955 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:57.551106 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6173801-39c7-4247-914f-825bfa351e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:57.552759 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6173801-39c7-4247-914f-825bfa351e1e admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:57.553337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6173801-39c7-4247-914f-825bfa351e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:57.553938 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6173801-39c7-4247-914f-825bfa351e1e admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:57.555528 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6173801-39c7-4247-914f-825bfa351e1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:57.559348 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6173801-39c7-4247-914f-825bfa351e1e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:57.583711 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1267/2544] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:00:57.591200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b09b37-75ae-477c-b634-ddb0c49c2459 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:57.611636 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b09b37-75ae-477c-b634-ddb0c49c2459 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:57.620620 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d2e40-fc89-461b-898d-33ec56154116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:57.622116 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d2e40-fc89-461b-898d-33ec56154116 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:57.622706 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d2e40-fc89-461b-898d-33ec56154116 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:57.623543 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d2e40-fc89-461b-898d-33ec56154116 admin admin] PATH_INFO: `/v3/projects/65fe21d36f9e4566a8671482ba842d25/users/266daf34c2554273af9057abcb756551/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:57.667099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655d2e40-fc89-461b-898d-33ec56154116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65fe21d36f9e4566a8671482ba842d25, user_id=266daf34c2554273af9057abcb756551, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:57.674535 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655d2e40-fc89-461b-898d-33ec56154116 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:57.730401 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1278/2545] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:00:57 2022] PUT /identity/v3/projects/65fe21d36f9e4566a8671482ba842d25/users/266daf34c2554273af9057abcb756551/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:57.754179 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4c573-ea00-45ef-a421-72a79212326c None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:57.756040 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4c573-ea00-45ef-a421-72a79212326c None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:57.756556 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4c573-ea00-45ef-a421-72a79212326c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:57.828965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-57a4c573-ea00-45ef-a421-72a79212326c None None] MFA Rules not processed for user `266daf34c2554273af9057abcb756551`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:57.831207 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-57a4c573-ea00-45ef-a421-72a79212326c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:58.060903 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1268/2546] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:58.073736 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d2e40-fc89-461b-898d-33ec56154116 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:58.075914 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-655d2e40-fc89-461b-898d-33ec56154116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:58.369657 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d2e40-fc89-461b-898d-33ec56154116 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:58.387573 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d29b5f-de5c-447a-bc2f-98b7ff4610d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:58.388439 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d29b5f-de5c-447a-bc2f-98b7ff4610d6 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:58.388728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d29b5f-de5c-447a-bc2f-98b7ff4610d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:58.389025 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d29b5f-de5c-447a-bc2f-98b7ff4610d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:58.411673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d29b5f-de5c-447a-bc2f-98b7ff4610d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:58.413488 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d29b5f-de5c-447a-bc2f-98b7ff4610d6 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:58.446959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1279/2547] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:00:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 09:00:58.472151 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4c573-ea00-45ef-a421-72a79212326c None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:58.710452 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:58.718947 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4c573-ea00-45ef-a421-72a79212326c None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:58.737341 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f698a3-521d-4e42-9e56-3db4461a79a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:58.738034 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f698a3-521d-4e42-9e56-3db4461a79a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:58.738317 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f698a3-521d-4e42-9e56-3db4461a79a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:58.738595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f698a3-521d-4e42-9e56-3db4461a79a8 admin admin] PATH_INFO: `/v3/users/266daf34c2554273af9057abcb756551` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:58.749484 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f698a3-521d-4e42-9e56-3db4461a79a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=266daf34c2554273af9057abcb756551)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:58.752662 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f698a3-521d-4e42-9e56-3db4461a79a8 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:58.843202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-75f698a3-521d-4e42-9e56-3db4461a79a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '266daf34c2554273af9057abcb756551'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:58.854694 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-75f698a3-521d-4e42-9e56-3db4461a79a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '266daf34c2554273af9057abcb756551'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:58.873079 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-75f698a3-521d-4e42-9e56-3db4461a79a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '266daf34c2554273af9057abcb756551'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:58.874585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-75f698a3-521d-4e42-9e56-3db4461a79a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '266daf34c2554273af9057abcb756551'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:58.875947 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-75f698a3-521d-4e42-9e56-3db4461a79a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '266daf34c2554273af9057abcb756551'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:00:58.916841 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1269/2548] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:00:58 2022] DELETE /identity/v3/users/266daf34c2554273af9057abcb756551 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:00:58.919189 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25842ef-973f-4965-a1b7-ec6b13b74c08 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:58.919478 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25842ef-973f-4965-a1b7-ec6b13b74c08 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:58.919755 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25842ef-973f-4965-a1b7-ec6b13b74c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:58.927381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d787e11-e217-466a-94b6-ac9f0a62f07b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:58.927381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d787e11-e217-466a-94b6-ac9f0a62f07b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:58.927696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d787e11-e217-466a-94b6-ac9f0a62f07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:58.956625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-b25842ef-973f-4965-a1b7-ec6b13b74c08 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:58.959190 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b25842ef-973f-4965-a1b7-ec6b13b74c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:59.014786 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-4d787e11-e217-466a-94b6-ac9f0a62f07b None None] MFA Rules not processed for user `b03c1f7daa07427888a86c7e8a69dd4d`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:00:59.017961 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4d787e11-e217-466a-94b6-ac9f0a62f07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:59.121160 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1280/2549] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 09:00:59.134461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25842ef-973f-4965-a1b7-ec6b13b74c08 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:59.137896 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b25842ef-973f-4965-a1b7-ec6b13b74c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:59.183969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:59.206600 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1270/2550] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:00:58 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:00:59.225616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d787e11-e217-466a-94b6-ac9f0a62f07b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:00:59.228095 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4d787e11-e217-466a-94b6-ac9f0a62f07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:59.269096 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25842ef-973f-4965-a1b7-ec6b13b74c08 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:59.276307 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645aba77-7e07-45f1-8b3e-6efb2ef7c471 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:59.277215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645aba77-7e07-45f1-8b3e-6efb2ef7c471 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:59.277762 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645aba77-7e07-45f1-8b3e-6efb2ef7c471 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:59.278086 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645aba77-7e07-45f1-8b3e-6efb2ef7c471 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:59.305271 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645aba77-7e07-45f1-8b3e-6efb2ef7c471 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:59.307348 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645aba77-7e07-45f1-8b3e-6efb2ef7c471 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:59.332527 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1281/2551] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:00:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:00:59.396521 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:00:59.439388 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d787e11-e217-466a-94b6-ac9f0a62f07b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:00:59.464520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7055d166-c36f-4260-b74d-a2de279bb507 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:00:59.466152 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055d166-c36f-4260-b74d-a2de279bb507 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:00:59.466937 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055d166-c36f-4260-b74d-a2de279bb507 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:00:59.474240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7055d166-c36f-4260-b74d-a2de279bb507 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:00:59.479040 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7055d166-c36f-4260-b74d-a2de279bb507 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:00:59.650452 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055d166-c36f-4260-b74d-a2de279bb507 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:00:59.659002 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7055d166-c36f-4260-b74d-a2de279bb507 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:00:59.692493 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1271/2552] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:00:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 09:01:00.097803 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645aba77-7e07-45f1-8b3e-6efb2ef7c471 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:00.100890 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-645aba77-7e07-45f1-8b3e-6efb2ef7c471 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:00.153172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645aba77-7e07-45f1-8b3e-6efb2ef7c471 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:00.166659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73b2ec-48e6-47fd-9113-553c263b06c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:00.168462 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73b2ec-48e6-47fd-9113-553c263b06c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:00.168462 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73b2ec-48e6-47fd-9113-553c263b06c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:00.168462 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73b2ec-48e6-47fd-9113-553c263b06c1 admin admin] PATH_INFO: `/v3/projects/65fe21d36f9e4566a8671482ba842d25` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:00.170664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73b2ec-48e6-47fd-9113-553c263b06c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65fe21d36f9e4566a8671482ba842d25)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:00.175100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73b2ec-48e6-47fd-9113-553c263b06c1 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:00.297397 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6d73b2ec-48e6-47fd-9113-553c263b06c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65fe21d36f9e4566a8671482ba842d25', 'request_id': 'req-6d73b2ec-48e6-47fd-9113-553c263b06c1'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:00.315512 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6d73b2ec-48e6-47fd-9113-553c263b06c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65fe21d36f9e4566a8671482ba842d25', 'request_id': 'req-6d73b2ec-48e6-47fd-9113-553c263b06c1'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:00.330659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6d73b2ec-48e6-47fd-9113-553c263b06c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65fe21d36f9e4566a8671482ba842d25', 'request_id': 'req-6d73b2ec-48e6-47fd-9113-553c263b06c1'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:00.353555 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6d73b2ec-48e6-47fd-9113-553c263b06c1 admin admin] The token cache is being invalidate because project 65fe21d36f9e4566a8671482ba842d25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:01:00.354484 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6d73b2ec-48e6-47fd-9113-553c263b06c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:00.362980 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6d73b2ec-48e6-47fd-9113-553c263b06c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:00.366230 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1282/2553] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:01:00 2022] DELETE /identity/v3/projects/65fe21d36f9e4566a8671482ba842d25 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:00.376898 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5edbe3-300a-417f-9926-f180a2450f4f None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:00.377340 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5edbe3-300a-417f-9926-f180a2450f4f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:00.377457 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5edbe3-300a-417f-9926-f180a2450f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:00.410808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-7d5edbe3-300a-417f-9926-f180a2450f4f None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:00.412875 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7d5edbe3-300a-417f-9926-f180a2450f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:00.541617 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1272/2554] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:00.556926 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04946f7-5abc-4e39-9acc-9e6e379b5046 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:00.557897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04946f7-5abc-4e39-9acc-9e6e379b5046 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:00.558670 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04946f7-5abc-4e39-9acc-9e6e379b5046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:00.604052 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-a04946f7-5abc-4e39-9acc-9e6e379b5046 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:00.611850 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a04946f7-5abc-4e39-9acc-9e6e379b5046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:00.627499 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1283/2555] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:00.633468 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5edbe3-300a-417f-9926-f180a2450f4f None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:00.658559 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5edbe3-300a-417f-9926-f180a2450f4f None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:00.669701 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32535d0c-6b92-486f-8f6c-b4f8e06d40c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:00.669701 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32535d0c-6b92-486f-8f6c-b4f8e06d40c1 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:00.670167 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32535d0c-6b92-486f-8f6c-b4f8e06d40c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:00.670648 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32535d0c-6b92-486f-8f6c-b4f8e06d40c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:00.672098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32535d0c-6b92-486f-8f6c-b4f8e06d40c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 09:01:00.672637 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32535d0c-6b92-486f-8f6c-b4f8e06d40c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:00.676474 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32535d0c-6b92-486f-8f6c-b4f8e06d40c1 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:00.694811 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1273/2556] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 09:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 09:01:00.702177 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04946f7-5abc-4e39-9acc-9e6e379b5046 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:00.720114 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04946f7-5abc-4e39-9acc-9e6e379b5046 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:00.746592 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e574f-8afb-415e-9b11-16cd0836ed0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:00.747641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e574f-8afb-415e-9b11-16cd0836ed0a admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:00.747977 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e574f-8afb-415e-9b11-16cd0836ed0a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:00.748272 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e574f-8afb-415e-9b11-16cd0836ed0a admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:00.749115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e574f-8afb-415e-9b11-16cd0836ed0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:00.752528 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e574f-8afb-415e-9b11-16cd0836ed0a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:00.802035 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1284/2557] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:01:00 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 09:01:00.804123 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32535d0c-6b92-486f-8f6c-b4f8e06d40c1 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:00.822827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32535d0c-6b92-486f-8f6c-b4f8e06d40c1 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:00.970864 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754cc42-cb93-44ad-8afa-81a67f984993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:00.972468 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754cc42-cb93-44ad-8afa-81a67f984993 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:00.972873 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754cc42-cb93-44ad-8afa-81a67f984993 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:00.973342 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754cc42-cb93-44ad-8afa-81a67f984993 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:00.974690 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754cc42-cb93-44ad-8afa-81a67f984993 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:00.978322 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754cc42-cb93-44ad-8afa-81a67f984993 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:01.032443 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1274/2558] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:01:00 2022] POST /identity/v3/users => generated 287 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:01:01.039895 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e574f-8afb-415e-9b11-16cd0836ed0a admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:01.086748 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e574f-8afb-415e-9b11-16cd0836ed0a admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:01.097351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b601c0-ab00-45d4-80b1-2f46d9930b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:01.098814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b601c0-ab00-45d4-80b1-2f46d9930b10 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:01.099346 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b601c0-ab00-45d4-80b1-2f46d9930b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:01.099841 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b601c0-ab00-45d4-80b1-2f46d9930b10 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:01.101089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b601c0-ab00-45d4-80b1-2f46d9930b10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:01.105009 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b601c0-ab00-45d4-80b1-2f46d9930b10 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:01.126527 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1285/2559] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:01.137107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754cc42-cb93-44ad-8afa-81a67f984993 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:01.170126 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754cc42-cb93-44ad-8afa-81a67f984993 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:01.185735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6da1b-b04f-4781-9387-6fba5c9b7775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:01.187310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6da1b-b04f-4781-9387-6fba5c9b7775 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:01.188458 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6da1b-b04f-4781-9387-6fba5c9b7775 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:01.189489 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6da1b-b04f-4781-9387-6fba5c9b7775 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:01.195601 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d6da1b-b04f-4781-9387-6fba5c9b7775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:01.198465 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d6da1b-b04f-4781-9387-6fba5c9b7775 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:01.222987 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1275/2560] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:01.228426 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b601c0-ab00-45d4-80b1-2f46d9930b10 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:01.246444 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6da1b-b04f-4781-9387-6fba5c9b7775 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:01.250157 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a8d6da1b-b04f-4781-9387-6fba5c9b7775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:01.265983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b601c0-ab00-45d4-80b1-2f46d9930b10 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:01.266314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:01.291437 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758d54d-8242-483a-8d69-15f9cf3b747e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:01.295701 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758d54d-8242-483a-8d69-15f9cf3b747e admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:01.296147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758d54d-8242-483a-8d69-15f9cf3b747e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:01.296457 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758d54d-8242-483a-8d69-15f9cf3b747e admin admin] PATH_INFO: `/v3/projects/84c69448ab6b4b448bc836893feae306/users/bab572bb38634e729aa267cb05a315c8/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:01.386865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758d54d-8242-483a-8d69-15f9cf3b747e admin admin] RBAC: Authorizing `identity:create_grant(project_id=84c69448ab6b4b448bc836893feae306, user_id=bab572bb38634e729aa267cb05a315c8, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:01.399779 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758d54d-8242-483a-8d69-15f9cf3b747e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:01.461960 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6da1b-b04f-4781-9387-6fba5c9b7775 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:01.464793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1286/2561] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:01:01 2022] PUT /identity/v3/projects/84c69448ab6b4b448bc836893feae306/users/bab572bb38634e729aa267cb05a315c8/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:01.466504 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aab68a-b215-4cf1-b541-8c7cf1e468bb None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:01.466797 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aab68a-b215-4cf1-b541-8c7cf1e468bb None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:01.467115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aab68a-b215-4cf1-b541-8c7cf1e468bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:01.533210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e8aab68a-b215-4cf1-b541-8c7cf1e468bb None None] MFA Rules not processed for user `bab572bb38634e729aa267cb05a315c8`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:01.536083 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e8aab68a-b215-4cf1-b541-8c7cf1e468bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:01.572699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6699063-1e95-4a21-9d62-ffa03c7a9d33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:01.575043 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6699063-1e95-4a21-9d62-ffa03c7a9d33 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:01.575757 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6699063-1e95-4a21-9d62-ffa03c7a9d33 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:01.576430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6699063-1e95-4a21-9d62-ffa03c7a9d33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:01.579890 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e6699063-1e95-4a21-9d62-ffa03c7a9d33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:01.624400 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:01.722565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1287/2562] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:01.740651 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aab68a-b215-4cf1-b541-8c7cf1e468bb None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:01.743388 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e8aab68a-b215-4cf1-b541-8c7cf1e468bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:01.766765 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6699063-1e95-4a21-9d62-ffa03c7a9d33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:01.770405 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6699063-1e95-4a21-9d62-ffa03c7a9d33 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:01.809438 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1276/2563] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:01:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 09:01:01.894947 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aab68a-b215-4cf1-b541-8c7cf1e468bb None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:01.901583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29bae5e-d065-446a-9330-cec2796b079f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:01.902565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29bae5e-d065-446a-9330-cec2796b079f service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:01.902863 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29bae5e-d065-446a-9330-cec2796b079f service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:01.903354 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29bae5e-d065-446a-9330-cec2796b079f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:01.925656 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29bae5e-d065-446a-9330-cec2796b079f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:01.928176 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29bae5e-d065-446a-9330-cec2796b079f service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:01.981747 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1288/2564] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:01:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 09:01:02.033087 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6699063-1e95-4a21-9d62-ffa03c7a9d33 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:02.047671 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6699063-1e95-4a21-9d62-ffa03c7a9d33 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:02.055651 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627ceca-5bce-4362-803a-319fbb163f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:02.056926 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627ceca-5bce-4362-803a-319fbb163f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:02.058073 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627ceca-5bce-4362-803a-319fbb163f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:02.058426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627ceca-5bce-4362-803a-319fbb163f10 admin admin] PATH_INFO: `/v3/users/bab572bb38634e729aa267cb05a315c8` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:02.060559 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e627ceca-5bce-4362-803a-319fbb163f10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bab572bb38634e729aa267cb05a315c8)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:02.064108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e627ceca-5bce-4362-803a-319fbb163f10 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:02.174695 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e627ceca-5bce-4362-803a-319fbb163f10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bab572bb38634e729aa267cb05a315c8'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:02.190856 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e627ceca-5bce-4362-803a-319fbb163f10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bab572bb38634e729aa267cb05a315c8'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:02.202519 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e627ceca-5bce-4362-803a-319fbb163f10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bab572bb38634e729aa267cb05a315c8'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:02.203329 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e627ceca-5bce-4362-803a-319fbb163f10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bab572bb38634e729aa267cb05a315c8'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:02.203996 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e627ceca-5bce-4362-803a-319fbb163f10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bab572bb38634e729aa267cb05a315c8'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:02.207593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:02.252858 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1277/2565] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:01:02 2022] DELETE /identity/v3/users/bab572bb38634e729aa267cb05a315c8 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:02.260711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff92a60-693f-483b-9ef6-4587f87114fa None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:02.262257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff92a60-693f-483b-9ef6-4587f87114fa None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:02.262963 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff92a60-693f-483b-9ef6-4587f87114fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:02.299629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-8ff92a60-693f-483b-9ef6-4587f87114fa None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:02.302695 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8ff92a60-693f-483b-9ef6-4587f87114fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:02.453071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1289/2566] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 09:01:02.478387 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627ceca-5bce-4362-803a-319fbb163f10 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:02.481030 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e627ceca-5bce-4362-803a-319fbb163f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:02.605999 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627ceca-5bce-4362-803a-319fbb163f10 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:02.614813 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b5715-41f1-4fec-b7a3-1f387d54b9be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:02.616597 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b5715-41f1-4fec-b7a3-1f387d54b9be service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:02.617472 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b5715-41f1-4fec-b7a3-1f387d54b9be service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:02.618380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b5715-41f1-4fec-b7a3-1f387d54b9be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:02.650298 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b5715-41f1-4fec-b7a3-1f387d54b9be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:02.653678 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b5715-41f1-4fec-b7a3-1f387d54b9be service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:02.681153 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1278/2567] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:01:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:01:03.869879 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff92a60-693f-483b-9ef6-4587f87114fa None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:03.872362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8ff92a60-693f-483b-9ef6-4587f87114fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:03.922650 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff92a60-693f-483b-9ef6-4587f87114fa None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:03.938170 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3457c9-f37a-4061-b06e-eec8d1c34c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:03.939614 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3457c9-f37a-4061-b06e-eec8d1c34c5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:03.940371 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3457c9-f37a-4061-b06e-eec8d1c34c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:03.941036 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3457c9-f37a-4061-b06e-eec8d1c34c5c admin admin] PATH_INFO: `/v3/projects/84c69448ab6b4b448bc836893feae306` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:03.944168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3457c9-f37a-4061-b06e-eec8d1c34c5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=84c69448ab6b4b448bc836893feae306)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:03.949150 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3457c9-f37a-4061-b06e-eec8d1c34c5c admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:04.064119 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-af3457c9-f37a-4061-b06e-eec8d1c34c5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84c69448ab6b4b448bc836893feae306', 'request_id': 'req-af3457c9-f37a-4061-b06e-eec8d1c34c5c'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:04.080106 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-af3457c9-f37a-4061-b06e-eec8d1c34c5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84c69448ab6b4b448bc836893feae306', 'request_id': 'req-af3457c9-f37a-4061-b06e-eec8d1c34c5c'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:04.096307 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-af3457c9-f37a-4061-b06e-eec8d1c34c5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84c69448ab6b4b448bc836893feae306', 'request_id': 'req-af3457c9-f37a-4061-b06e-eec8d1c34c5c'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:04.111376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:04.137751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-af3457c9-f37a-4061-b06e-eec8d1c34c5c admin admin] The token cache is being invalidate because project 84c69448ab6b4b448bc836893feae306 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:01:04.138916 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-af3457c9-f37a-4061-b06e-eec8d1c34c5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:04.140836 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-af3457c9-f37a-4061-b06e-eec8d1c34c5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:04.150452 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1290/2568] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:01:03 2022] DELETE /identity/v3/projects/84c69448ab6b4b448bc836893feae306 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 09:01:04.152544 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db2673-a664-436e-931a-b5d2db12c9a6 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:04.153373 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db2673-a664-436e-931a-b5d2db12c9a6 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:04.153570 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db2673-a664-436e-931a-b5d2db12c9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:04.188439 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-52db2673-a664-436e-931a-b5d2db12c9a6 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:04.190493 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-52db2673-a664-436e-931a-b5d2db12c9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:04.412133 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1279/2569] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 09:01:04.428909 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c9cdf-7c38-4480-944f-62a45bbf3fcf None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:04.428909 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c9cdf-7c38-4480-944f-62a45bbf3fcf None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:04.428909 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c9cdf-7c38-4480-944f-62a45bbf3fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:04.475799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-bc5c9cdf-7c38-4480-944f-62a45bbf3fcf None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:04.477298 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-bc5c9cdf-7c38-4480-944f-62a45bbf3fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:04.487049 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1291/2570] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:04.492518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db2673-a664-436e-931a-b5d2db12c9a6 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:04.516425 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db2673-a664-436e-931a-b5d2db12c9a6 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:04.527497 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb47d47-36f4-4e51-be71-80380fc4d6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:04.528663 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb47d47-36f4-4e51-be71-80380fc4d6fd admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:04.529174 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb47d47-36f4-4e51-be71-80380fc4d6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:04.529649 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb47d47-36f4-4e51-be71-80380fc4d6fd admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:04.531761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb47d47-36f4-4e51-be71-80380fc4d6fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 09:01:04.532308 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb47d47-36f4-4e51-be71-80380fc4d6fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:04.537503 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb47d47-36f4-4e51-be71-80380fc4d6fd admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:04.581022 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1280/2571] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 09:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 09:01:04.581580 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c9cdf-7c38-4480-944f-62a45bbf3fcf None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:04.593365 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c9cdf-7c38-4480-944f-62a45bbf3fcf None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:04.607364 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b5f8d-c5ba-4c69-b01a-f4672bcf59ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:04.608271 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b5f8d-c5ba-4c69-b01a-f4672bcf59ee admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:04.608610 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b5f8d-c5ba-4c69-b01a-f4672bcf59ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:04.608986 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b5f8d-c5ba-4c69-b01a-f4672bcf59ee admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:04.609682 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b5f8d-c5ba-4c69-b01a-f4672bcf59ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:04.612399 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b5f8d-c5ba-4c69-b01a-f4672bcf59ee admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:04.697077 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1292/2572] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:01:04 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:01:04.702426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb47d47-36f4-4e51-be71-80380fc4d6fd admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:04.721984 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb47d47-36f4-4e51-be71-80380fc4d6fd admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:04.872263 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0cb33-10b1-40ba-a1a5-1ae8f6f48562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:04.873357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0cb33-10b1-40ba-a1a5-1ae8f6f48562 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:04.873857 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0cb33-10b1-40ba-a1a5-1ae8f6f48562 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:04.874359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0cb33-10b1-40ba-a1a5-1ae8f6f48562 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:04.875790 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0cb33-10b1-40ba-a1a5-1ae8f6f48562 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:04.878913 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0cb33-10b1-40ba-a1a5-1ae8f6f48562 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:04.924856 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1281/2573] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:01:04 2022] POST /identity/v3/users => generated 292 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:01:04.926275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b5f8d-c5ba-4c69-b01a-f4672bcf59ee admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:04.946764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b5f8d-c5ba-4c69-b01a-f4672bcf59ee admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:04.955429 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a201277-99dd-4f40-a9f7-9ba14e441bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:04.956536 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a201277-99dd-4f40-a9f7-9ba14e441bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:04.957894 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a201277-99dd-4f40-a9f7-9ba14e441bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:04.958468 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a201277-99dd-4f40-a9f7-9ba14e441bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:04.959633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a201277-99dd-4f40-a9f7-9ba14e441bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:04.962343 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a201277-99dd-4f40-a9f7-9ba14e441bd6 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:04.980295 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1293/2574] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:04 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 09:01:04.989369 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0cb33-10b1-40ba-a1a5-1ae8f6f48562 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:04.998407 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:05.006577 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0cb33-10b1-40ba-a1a5-1ae8f6f48562 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:05.015909 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7a121-cad1-48f3-a45b-1813540036af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:05.017894 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7a121-cad1-48f3-a45b-1813540036af admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:05.019275 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7a121-cad1-48f3-a45b-1813540036af admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:05.019737 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7a121-cad1-48f3-a45b-1813540036af admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:05.020877 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec7a121-cad1-48f3-a45b-1813540036af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:05.023754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec7a121-cad1-48f3-a45b-1813540036af admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:05.061492 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1282/2575] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:04 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:05.066057 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a201277-99dd-4f40-a9f7-9ba14e441bd6 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:05.083959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a201277-99dd-4f40-a9f7-9ba14e441bd6 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:05.091367 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402441d4-19bf-4e93-868c-b82101a6b769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:05.092725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402441d4-19bf-4e93-868c-b82101a6b769 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:05.093225 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402441d4-19bf-4e93-868c-b82101a6b769 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:05.094136 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402441d4-19bf-4e93-868c-b82101a6b769 admin admin] PATH_INFO: `/v3/projects/eed42d5a00d44ae3af1d74159e20c616/users/d5e175245f2e48c79d5f82dbaf4f1cf4/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:05.119507 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402441d4-19bf-4e93-868c-b82101a6b769 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eed42d5a00d44ae3af1d74159e20c616, user_id=d5e175245f2e48c79d5f82dbaf4f1cf4, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:05.124149 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402441d4-19bf-4e93-868c-b82101a6b769 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:05.147714 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1294/2576] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:01:05 2022] PUT /identity/v3/projects/eed42d5a00d44ae3af1d74159e20c616/users/d5e175245f2e48c79d5f82dbaf4f1cf4/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:05.155411 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c871a85-e889-4b0b-8bf3-fa6025de098f None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:05.155890 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c871a85-e889-4b0b-8bf3-fa6025de098f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:05.156240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c871a85-e889-4b0b-8bf3-fa6025de098f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:05.188339 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-9c871a85-e889-4b0b-8bf3-fa6025de098f None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:05.189808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9c871a85-e889-4b0b-8bf3-fa6025de098f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:05.392752 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1283/2577] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:05.405965 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402441d4-19bf-4e93-868c-b82101a6b769 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:05.411042 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-402441d4-19bf-4e93-868c-b82101a6b769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:05.607399 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402441d4-19bf-4e93-868c-b82101a6b769 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:05.614440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aded23-5352-42ab-be74-ea6ad411bcac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:05.615776 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aded23-5352-42ab-be74-ea6ad411bcac service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:05.616236 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aded23-5352-42ab-be74-ea6ad411bcac service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:05.616706 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aded23-5352-42ab-be74-ea6ad411bcac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:05.636833 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aded23-5352-42ab-be74-ea6ad411bcac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:05.639769 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aded23-5352-42ab-be74-ea6ad411bcac service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:05.662028 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1295/2578] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:01:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:01:07.299701 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7823990f-a8e1-4484-9d50-841fe9d58d2b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:07.300261 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7823990f-a8e1-4484-9d50-841fe9d58d2b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:07.300261 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7823990f-a8e1-4484-9d50-841fe9d58d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:07.393913 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-7823990f-a8e1-4484-9d50-841fe9d58d2b None None] MFA Rules not processed for user `d5e175245f2e48c79d5f82dbaf4f1cf4`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:07.398708 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7823990f-a8e1-4484-9d50-841fe9d58d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:07.548504 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1284/2579] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:07.552302 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aded23-5352-42ab-be74-ea6ad411bcac service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:07.556067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-79aded23-5352-42ab-be74-ea6ad411bcac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:07.683452 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aded23-5352-42ab-be74-ea6ad411bcac service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:07.690451 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df1aae-4c90-4176-a593-9b7e5e895abf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:07.692851 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df1aae-4c90-4176-a593-9b7e5e895abf service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:07.693246 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df1aae-4c90-4176-a593-9b7e5e895abf service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:07.694192 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df1aae-4c90-4176-a593-9b7e5e895abf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:07.711586 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df1aae-4c90-4176-a593-9b7e5e895abf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:07.713399 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df1aae-4c90-4176-a593-9b7e5e895abf service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:07.730562 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1296/2580] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:01:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:01:08.442470 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7823990f-a8e1-4484-9d50-841fe9d58d2b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:08.444083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7823990f-a8e1-4484-9d50-841fe9d58d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:08.460189 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7823990f-a8e1-4484-9d50-841fe9d58d2b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:08.466438 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2e472-9831-4bb3-bf59-792b0de58d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:08.467163 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2e472-9831-4bb3-bf59-792b0de58d11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:08.467463 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2e472-9831-4bb3-bf59-792b0de58d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:08.467731 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2e472-9831-4bb3-bf59-792b0de58d11 admin admin] PATH_INFO: `/v3/users/63e1c5a24be04ca4abd5d5109c5763f8` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:08.468872 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b2e472-9831-4bb3-bf59-792b0de58d11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63e1c5a24be04ca4abd5d5109c5763f8)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:08.470855 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b2e472-9831-4bb3-bf59-792b0de58d11 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:08.525734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-60b2e472-9831-4bb3-bf59-792b0de58d11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63e1c5a24be04ca4abd5d5109c5763f8'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:08.540803 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-60b2e472-9831-4bb3-bf59-792b0de58d11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63e1c5a24be04ca4abd5d5109c5763f8'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:08.549939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-60b2e472-9831-4bb3-bf59-792b0de58d11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63e1c5a24be04ca4abd5d5109c5763f8'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:08.550838 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-60b2e472-9831-4bb3-bf59-792b0de58d11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63e1c5a24be04ca4abd5d5109c5763f8'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:08.551581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-60b2e472-9831-4bb3-bf59-792b0de58d11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63e1c5a24be04ca4abd5d5109c5763f8'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:08.565085 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:08.583345 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1285/2581] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:01:08 2022] DELETE /identity/v3/users/63e1c5a24be04ca4abd5d5109c5763f8 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 09:01:08.770560 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df1aae-4c90-4176-a593-9b7e5e895abf service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:08.783232 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df1aae-4c90-4176-a593-9b7e5e895abf service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:08.832431 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:08.852686 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a24863-b80e-4811-af7c-f0eaf6d2a38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:08.853321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a24863-b80e-4811-af7c-f0eaf6d2a38c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:08.853599 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a24863-b80e-4811-af7c-f0eaf6d2a38c admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:08.853876 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a24863-b80e-4811-af7c-f0eaf6d2a38c admin admin] PATH_INFO: `/v3/projects/8ed7993eba324f94a1b0f7fc23401f0f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:08.855039 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a24863-b80e-4811-af7c-f0eaf6d2a38c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ed7993eba324f94a1b0f7fc23401f0f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:08.857167 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a24863-b80e-4811-af7c-f0eaf6d2a38c admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:08.913357 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-64a24863-b80e-4811-af7c-f0eaf6d2a38c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ed7993eba324f94a1b0f7fc23401f0f', 'request_id': 'req-64a24863-b80e-4811-af7c-f0eaf6d2a38c'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:08.920464 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-64a24863-b80e-4811-af7c-f0eaf6d2a38c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ed7993eba324f94a1b0f7fc23401f0f', 'request_id': 'req-64a24863-b80e-4811-af7c-f0eaf6d2a38c'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:08.938131 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-64a24863-b80e-4811-af7c-f0eaf6d2a38c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ed7993eba324f94a1b0f7fc23401f0f', 'request_id': 'req-64a24863-b80e-4811-af7c-f0eaf6d2a38c'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:08.949902 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-64a24863-b80e-4811-af7c-f0eaf6d2a38c admin admin] The token cache is being invalidate because project 8ed7993eba324f94a1b0f7fc23401f0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:01:08.950183 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-64a24863-b80e-4811-af7c-f0eaf6d2a38c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:08.950819 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-64a24863-b80e-4811-af7c-f0eaf6d2a38c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:08.952353 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1297/2582] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:01:08 2022] DELETE /identity/v3/projects/8ed7993eba324f94a1b0f7fc23401f0f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:08.957828 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce7dfb-4ac5-444b-ad80-d1dcc7384073 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:08.958256 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce7dfb-4ac5-444b-ad80-d1dcc7384073 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:08.958605 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce7dfb-4ac5-444b-ad80-d1dcc7384073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:08.981158 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-45ce7dfb-4ac5-444b-ad80-d1dcc7384073 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:08.982369 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-45ce7dfb-4ac5-444b-ad80-d1dcc7384073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:09.054427 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1286/2583] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 09:01:09.063320 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfb605-583e-4888-9350-187ce2b4b525 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:09.063584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfb605-583e-4888-9350-187ce2b4b525 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:09.063905 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfb605-583e-4888-9350-187ce2b4b525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:09.089763 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-fedfb605-583e-4888-9350-187ce2b4b525 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:09.091451 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-fedfb605-583e-4888-9350-187ce2b4b525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:09.103719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1298/2584] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:09.108357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce7dfb-4ac5-444b-ad80-d1dcc7384073 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:09.119088 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce7dfb-4ac5-444b-ad80-d1dcc7384073 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:09.124081 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda352a-9d0a-4466-8b6f-3c1746acdd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:09.124693 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda352a-9d0a-4466-8b6f-3c1746acdd8a admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:09.124963 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda352a-9d0a-4466-8b6f-3c1746acdd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:09.125233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda352a-9d0a-4466-8b6f-3c1746acdd8a admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:09.125994 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda352a-9d0a-4466-8b6f-3c1746acdd8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 09:01:09.126296 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda352a-9d0a-4466-8b6f-3c1746acdd8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:09.128167 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda352a-9d0a-4466-8b6f-3c1746acdd8a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:09.141586 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1287/2585] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 09:01:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 09:01:09.146460 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfb605-583e-4888-9350-187ce2b4b525 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:09.156960 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfb605-583e-4888-9350-187ce2b4b525 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:09.164116 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c6add-368a-47f3-91a0-655fec756b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:09.164670 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c6add-368a-47f3-91a0-655fec756b38 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:09.164943 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c6add-368a-47f3-91a0-655fec756b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:09.165212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c6add-368a-47f3-91a0-655fec756b38 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:09.166053 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c6add-368a-47f3-91a0-655fec756b38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:09.168417 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c6add-368a-47f3-91a0-655fec756b38 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:09.198813 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1299/2586] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:01:09 2022] POST /identity/v3/projects => generated 405 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:01:09.203731 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda352a-9d0a-4466-8b6f-3c1746acdd8a admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:09.216247 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda352a-9d0a-4466-8b6f-3c1746acdd8a admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:09.281850 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508282cf-60a1-4c3b-8afc-39ddca19f398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:09.282466 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508282cf-60a1-4c3b-8afc-39ddca19f398 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:09.282744 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508282cf-60a1-4c3b-8afc-39ddca19f398 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:09.283093 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508282cf-60a1-4c3b-8afc-39ddca19f398 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:09.283738 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508282cf-60a1-4c3b-8afc-39ddca19f398 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:09.285693 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508282cf-60a1-4c3b-8afc-39ddca19f398 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:09.310793 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1288/2587] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:01:09 2022] POST /identity/v3/users => generated 303 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:01:09.316564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c6add-368a-47f3-91a0-655fec756b38 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:09.328099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c6add-368a-47f3-91a0-655fec756b38 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:09.333262 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875b648-ec48-47cf-886c-01c4dd5163fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:09.333947 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875b648-ec48-47cf-886c-01c4dd5163fb admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:09.334239 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875b648-ec48-47cf-886c-01c4dd5163fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:09.334567 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875b648-ec48-47cf-886c-01c4dd5163fb admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:09.335323 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875b648-ec48-47cf-886c-01c4dd5163fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:09.337024 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875b648-ec48-47cf-886c-01c4dd5163fb admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:09.366791 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1300/2588] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:09.370144 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06dea6-30f5-44a9-99c8-586dd9e491cc None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:09.371888 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06dea6-30f5-44a9-99c8-586dd9e491cc None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:09.371888 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06dea6-30f5-44a9-99c8-586dd9e491cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:09.373706 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875b648-ec48-47cf-886c-01c4dd5163fb admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:09.385787 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875b648-ec48-47cf-886c-01c4dd5163fb admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:09.395240 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1626eeb0-1d07-41c5-a450-56858a73e58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:09.396004 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626eeb0-1d07-41c5-a450-56858a73e58a admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:09.396375 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626eeb0-1d07-41c5-a450-56858a73e58a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:09.396728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626eeb0-1d07-41c5-a450-56858a73e58a admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:09.397468 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1626eeb0-1d07-41c5-a450-56858a73e58a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:09.399550 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1626eeb0-1d07-41c5-a450-56858a73e58a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:09.404249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-fb06dea6-30f5-44a9-99c8-586dd9e491cc None None] MFA Rules not processed for user `d5e175245f2e48c79d5f82dbaf4f1cf4`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:09.410066 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-fb06dea6-30f5-44a9-99c8-586dd9e491cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:09.417071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1301/2589] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:09.418540 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1626eeb0-1d07-41c5-a450-56858a73e58a admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:09.436467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1626eeb0-1d07-41c5-a450-56858a73e58a admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:09.441422 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728ef39-39eb-4ace-9c87-f0910b94f642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:09.442094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728ef39-39eb-4ace-9c87-f0910b94f642 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:09.442479 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728ef39-39eb-4ace-9c87-f0910b94f642 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:09.442804 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728ef39-39eb-4ace-9c87-f0910b94f642 admin admin] PATH_INFO: `/v3/projects/304f7f7bb0da408490da0031398edb91/users/c4f5e860a84e48588b9f1966691f6187/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:09.460348 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728ef39-39eb-4ace-9c87-f0910b94f642 admin admin] RBAC: Authorizing `identity:create_grant(project_id=304f7f7bb0da408490da0031398edb91, user_id=c4f5e860a84e48588b9f1966691f6187, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:09.463220 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e728ef39-39eb-4ace-9c87-f0910b94f642 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:09.494987 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1302/2590] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:01:09 2022] PUT /identity/v3/projects/304f7f7bb0da408490da0031398edb91/users/c4f5e860a84e48588b9f1966691f6187/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 09:01:09.495257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1289/2591] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 09:01:09.502760 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d6bae-7b22-4af5-9b01-a31681517820 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:09.503158 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d6bae-7b22-4af5-9b01-a31681517820 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:09.503463 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d6bae-7b22-4af5-9b01-a31681517820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:09.536254 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-979d6bae-7b22-4af5-9b01-a31681517820 None None] MFA Rules not processed for user `c4f5e860a84e48588b9f1966691f6187`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:09.537581 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-979d6bae-7b22-4af5-9b01-a31681517820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:09.594743 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:09.643196 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1303/2592] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:09 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:09.644728 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06dea6-30f5-44a9-99c8-586dd9e491cc None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:09.656803 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06dea6-30f5-44a9-99c8-586dd9e491cc None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:09.669961 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:09.723119 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a30fdf5-a82a-4cac-8292-1e3638f3aabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:09.723738 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30fdf5-a82a-4cac-8292-1e3638f3aabc admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:09.724015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30fdf5-a82a-4cac-8292-1e3638f3aabc admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:09.724284 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30fdf5-a82a-4cac-8292-1e3638f3aabc admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:09.724912 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a30fdf5-a82a-4cac-8292-1e3638f3aabc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:09.726946 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a30fdf5-a82a-4cac-8292-1e3638f3aabc admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:09.759015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1290/2593] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:01:09 2022] POST /identity/v3/projects => generated 403 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:01:09.763147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d6bae-7b22-4af5-9b01-a31681517820 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:09.777875 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d6bae-7b22-4af5-9b01-a31681517820 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:09.872941 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab7622-a8d0-49ee-b31a-8160d6a9bd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:09.873752 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab7622-a8d0-49ee-b31a-8160d6a9bd64 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:09.874226 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab7622-a8d0-49ee-b31a-8160d6a9bd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:09.875412 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab7622-a8d0-49ee-b31a-8160d6a9bd64 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:09.876348 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab7622-a8d0-49ee-b31a-8160d6a9bd64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:09.878561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab7622-a8d0-49ee-b31a-8160d6a9bd64 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:09.917483 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1304/2594] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:01:09 2022] POST /identity/v3/users => generated 308 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:01:09.922741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a30fdf5-a82a-4cac-8292-1e3638f3aabc admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:09.940849 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a30fdf5-a82a-4cac-8292-1e3638f3aabc admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:09.947410 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b63c80-8e88-41eb-840c-6ca230651242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:09.948703 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b63c80-8e88-41eb-840c-6ca230651242 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:09.949018 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b63c80-8e88-41eb-840c-6ca230651242 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:09.949267 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b63c80-8e88-41eb-840c-6ca230651242 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:09.950127 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b63c80-8e88-41eb-840c-6ca230651242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:09.952046 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b63c80-8e88-41eb-840c-6ca230651242 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:09.968527 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1291/2595] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:09.974013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab7622-a8d0-49ee-b31a-8160d6a9bd64 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:09.987097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab7622-a8d0-49ee-b31a-8160d6a9bd64 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:09.993243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b0762-4038-47e8-8e71-967bc9457612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:09.994250 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b0762-4038-47e8-8e71-967bc9457612 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:09.994631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b0762-4038-47e8-8e71-967bc9457612 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:09.995107 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b0762-4038-47e8-8e71-967bc9457612 admin admin] PATH_INFO: `/v3/projects/ba230104946847449dc415b79c740743/users/ecf18c4f572d47708987cc7d5205ff86/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:10.018026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b0762-4038-47e8-8e71-967bc9457612 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba230104946847449dc415b79c740743, user_id=ecf18c4f572d47708987cc7d5205ff86, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:10.021663 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b0762-4038-47e8-8e71-967bc9457612 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:10.041210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1305/2596] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:01:09 2022] PUT /identity/v3/projects/ba230104946847449dc415b79c740743/users/ecf18c4f572d47708987cc7d5205ff86/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 09:01:10.045695 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b63c80-8e88-41eb-840c-6ca230651242 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:10.060779 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b63c80-8e88-41eb-840c-6ca230651242 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:10.211964 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55faa3a7-9637-4481-b216-5effd8d3f37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:10.212744 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55faa3a7-9637-4481-b216-5effd8d3f37f admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:10.213264 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55faa3a7-9637-4481-b216-5effd8d3f37f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:10.213729 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55faa3a7-9637-4481-b216-5effd8d3f37f admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:10.214538 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55faa3a7-9637-4481-b216-5effd8d3f37f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:10.216543 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55faa3a7-9637-4481-b216-5effd8d3f37f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:10.237807 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1292/2597] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:10 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:10.242185 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b0762-4038-47e8-8e71-967bc9457612 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:10.268847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b0762-4038-47e8-8e71-967bc9457612 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:10.276035 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d92e1-9412-4fde-b98e-2fa3048e3e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:10.277272 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d92e1-9412-4fde-b98e-2fa3048e3e21 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:10.277656 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d92e1-9412-4fde-b98e-2fa3048e3e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:10.279970 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d92e1-9412-4fde-b98e-2fa3048e3e21 admin admin] PATH_INFO: `/v3/projects/ba230104946847449dc415b79c740743/users/ecf18c4f572d47708987cc7d5205ff86/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:10.297706 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d92e1-9412-4fde-b98e-2fa3048e3e21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba230104946847449dc415b79c740743, user_id=ecf18c4f572d47708987cc7d5205ff86, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:10.301820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d92e1-9412-4fde-b98e-2fa3048e3e21 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:10.315517 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1306/2598] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:01:10 2022] PUT /identity/v3/projects/ba230104946847449dc415b79c740743/users/ecf18c4f572d47708987cc7d5205ff86/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:10.328092 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538790c-8392-47cd-8a5a-29441a66611e None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:10.328379 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538790c-8392-47cd-8a5a-29441a66611e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:10.328804 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538790c-8392-47cd-8a5a-29441a66611e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:10.372222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-f538790c-8392-47cd-8a5a-29441a66611e None None] MFA Rules not processed for user `ecf18c4f572d47708987cc7d5205ff86`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:10.373548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f538790c-8392-47cd-8a5a-29441a66611e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:10.472717 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1293/2599] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:10 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:10.486593 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d92e1-9412-4fde-b98e-2fa3048e3e21 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:10.488380 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f72d92e1-9412-4fde-b98e-2fa3048e3e21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:10.572170 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d92e1-9412-4fde-b98e-2fa3048e3e21 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:10.576330 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d75de-2356-4671-8457-692ed5b96d58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:10.577151 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d75de-2356-4671-8457-692ed5b96d58 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:10.577933 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d75de-2356-4671-8457-692ed5b96d58 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:10.577933 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d75de-2356-4671-8457-692ed5b96d58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:10.588445 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d75de-2356-4671-8457-692ed5b96d58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:10.590089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d75de-2356-4671-8457-692ed5b96d58 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:10.605456 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1307/2600] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:01:10 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:01:10.790094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f538790c-8392-47cd-8a5a-29441a66611e None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:10.818040 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:10.884414 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f538790c-8392-47cd-8a5a-29441a66611e None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:10.890246 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0aabb-11ef-40e5-856f-e62293cb0197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:10.890980 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0aabb-11ef-40e5-856f-e62293cb0197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:10.891517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0aabb-11ef-40e5-856f-e62293cb0197 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:10.891832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0aabb-11ef-40e5-856f-e62293cb0197 admin admin] PATH_INFO: `/v3/users/c4f5e860a84e48588b9f1966691f6187` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:10.893163 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0aabb-11ef-40e5-856f-e62293cb0197 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4f5e860a84e48588b9f1966691f6187)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:10.895125 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0aabb-11ef-40e5-856f-e62293cb0197 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:10.953396 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c4f0aabb-11ef-40e5-856f-e62293cb0197 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4f5e860a84e48588b9f1966691f6187'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:10.964480 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c4f0aabb-11ef-40e5-856f-e62293cb0197 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4f5e860a84e48588b9f1966691f6187'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:10.975985 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c4f0aabb-11ef-40e5-856f-e62293cb0197 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4f5e860a84e48588b9f1966691f6187'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:10.976839 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c4f0aabb-11ef-40e5-856f-e62293cb0197 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4f5e860a84e48588b9f1966691f6187'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:10.977529 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-c4f0aabb-11ef-40e5-856f-e62293cb0197 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4f5e860a84e48588b9f1966691f6187'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:11.005828 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1294/2601] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:01:10 2022] DELETE /identity/v3/users/c4f5e860a84e48588b9f1966691f6187 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:11.012008 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839025db-a42c-4405-a4b3-baaffb56fc2f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:11.013084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839025db-a42c-4405-a4b3-baaffb56fc2f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:11.013084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839025db-a42c-4405-a4b3-baaffb56fc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:11.040097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-839025db-a42c-4405-a4b3-baaffb56fc2f None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:11.041556 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-839025db-a42c-4405-a4b3-baaffb56fc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:11.079522 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:11.109952 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1308/2602] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:11.120129 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0aabb-11ef-40e5-856f-e62293cb0197 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:11.121704 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c4f0aabb-11ef-40e5-856f-e62293cb0197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:11.187761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0aabb-11ef-40e5-856f-e62293cb0197 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:11.191523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e332a-a5cb-4681-9051-6c68b7afa5f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:11.192143 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e332a-a5cb-4681-9051-6c68b7afa5f4 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:11.192419 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e332a-a5cb-4681-9051-6c68b7afa5f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:11.192688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e332a-a5cb-4681-9051-6c68b7afa5f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:11.207440 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e332a-a5cb-4681-9051-6c68b7afa5f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:11.209008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e332a-a5cb-4681-9051-6c68b7afa5f4 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:11.220502 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1295/2603] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:01:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:01:11.556394 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839025db-a42c-4405-a4b3-baaffb56fc2f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:11.560785 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-839025db-a42c-4405-a4b3-baaffb56fc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:11.582848 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839025db-a42c-4405-a4b3-baaffb56fc2f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:11.603758 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a2530-1e3d-4f5a-8002-85794bdd11c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:11.604638 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a2530-1e3d-4f5a-8002-85794bdd11c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:11.604921 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a2530-1e3d-4f5a-8002-85794bdd11c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:11.605187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a2530-1e3d-4f5a-8002-85794bdd11c8 admin admin] PATH_INFO: `/v3/projects/304f7f7bb0da408490da0031398edb91` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:11.607016 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a2530-1e3d-4f5a-8002-85794bdd11c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=304f7f7bb0da408490da0031398edb91)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:11.609777 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a2530-1e3d-4f5a-8002-85794bdd11c8 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:11.698850 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2e1a2530-1e3d-4f5a-8002-85794bdd11c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '304f7f7bb0da408490da0031398edb91', 'request_id': 'req-2e1a2530-1e3d-4f5a-8002-85794bdd11c8'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:11.707381 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2e1a2530-1e3d-4f5a-8002-85794bdd11c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '304f7f7bb0da408490da0031398edb91', 'request_id': 'req-2e1a2530-1e3d-4f5a-8002-85794bdd11c8'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:11.721276 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2e1a2530-1e3d-4f5a-8002-85794bdd11c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '304f7f7bb0da408490da0031398edb91', 'request_id': 'req-2e1a2530-1e3d-4f5a-8002-85794bdd11c8'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:11.746716 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2e1a2530-1e3d-4f5a-8002-85794bdd11c8 admin admin] The token cache is being invalidate because project 304f7f7bb0da408490da0031398edb91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:01:11.747084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2e1a2530-1e3d-4f5a-8002-85794bdd11c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:11.749103 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2e1a2530-1e3d-4f5a-8002-85794bdd11c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:11.754111 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1309/2604] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:01:11 2022] DELETE /identity/v3/projects/304f7f7bb0da408490da0031398edb91 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:11.755853 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e332a-a5cb-4681-9051-6c68b7afa5f4 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:11.767131 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e332a-a5cb-4681-9051-6c68b7afa5f4 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:11.825962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:11.875132 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a301b-515d-4445-ad29-262634ad9839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:11.875353 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a301b-515d-4445-ad29-262634ad9839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:11.875707 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a301b-515d-4445-ad29-262634ad9839 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:11.876224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a301b-515d-4445-ad29-262634ad9839 admin admin] PATH_INFO: `/v3/users/ecf18c4f572d47708987cc7d5205ff86` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:11.877568 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a301b-515d-4445-ad29-262634ad9839 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecf18c4f572d47708987cc7d5205ff86)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:11.880152 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a301b-515d-4445-ad29-262634ad9839 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:11.945203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-794a301b-515d-4445-ad29-262634ad9839 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecf18c4f572d47708987cc7d5205ff86'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:11.953165 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-794a301b-515d-4445-ad29-262634ad9839 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecf18c4f572d47708987cc7d5205ff86'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:11.963306 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-794a301b-515d-4445-ad29-262634ad9839 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecf18c4f572d47708987cc7d5205ff86'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:11.964061 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-794a301b-515d-4445-ad29-262634ad9839 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecf18c4f572d47708987cc7d5205ff86'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:11.964691 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-794a301b-515d-4445-ad29-262634ad9839 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecf18c4f572d47708987cc7d5205ff86'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:11.984943 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1296/2605] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:01:11 2022] DELETE /identity/v3/users/ecf18c4f572d47708987cc7d5205ff86 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:12.526174 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a2530-1e3d-4f5a-8002-85794bdd11c8 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:12.528456 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2e1a2530-1e3d-4f5a-8002-85794bdd11c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:12.553668 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a2530-1e3d-4f5a-8002-85794bdd11c8 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:12.671013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c04a625-68a4-4790-ab60-e89dae3f1da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:12.671713 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04a625-68a4-4790-ab60-e89dae3f1da8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:12.672047 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04a625-68a4-4790-ab60-e89dae3f1da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:12.672320 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04a625-68a4-4790-ab60-e89dae3f1da8 admin admin] PATH_INFO: `/v3/projects/ba230104946847449dc415b79c740743` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:12.674290 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c04a625-68a4-4790-ab60-e89dae3f1da8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba230104946847449dc415b79c740743)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:12.677102 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c04a625-68a4-4790-ab60-e89dae3f1da8 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:12.743102 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-3c04a625-68a4-4790-ab60-e89dae3f1da8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba230104946847449dc415b79c740743', 'request_id': 'req-3c04a625-68a4-4790-ab60-e89dae3f1da8'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:12.750118 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-3c04a625-68a4-4790-ab60-e89dae3f1da8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba230104946847449dc415b79c740743', 'request_id': 'req-3c04a625-68a4-4790-ab60-e89dae3f1da8'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:12.760965 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-3c04a625-68a4-4790-ab60-e89dae3f1da8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba230104946847449dc415b79c740743', 'request_id': 'req-3c04a625-68a4-4790-ab60-e89dae3f1da8'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:12.763117 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:12.791923 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-3c04a625-68a4-4790-ab60-e89dae3f1da8 admin admin] The token cache is being invalidate because project ba230104946847449dc415b79c740743 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:01:12.792410 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-3c04a625-68a4-4790-ab60-e89dae3f1da8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:12.793516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-3c04a625-68a4-4790-ab60-e89dae3f1da8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:12.796715 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1310/2606] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:01:12 2022] DELETE /identity/v3/projects/ba230104946847449dc415b79c740743 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:12.802935 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72ce8c-8ee7-4b4c-a39c-89d8d444ee82 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:12.803259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72ce8c-8ee7-4b4c-a39c-89d8d444ee82 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:12.803259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72ce8c-8ee7-4b4c-a39c-89d8d444ee82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:12.828297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-0f72ce8c-8ee7-4b4c-a39c-89d8d444ee82 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:12.830432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-0f72ce8c-8ee7-4b4c-a39c-89d8d444ee82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:12.926865 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1297/2607] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 09:01:12.936442 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd442f-6163-4efa-9414-6ffbad5b63df None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:12.936887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd442f-6163-4efa-9414-6ffbad5b63df None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:12.937625 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd442f-6163-4efa-9414-6ffbad5b63df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:12.979997 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-88bd442f-6163-4efa-9414-6ffbad5b63df None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:12.981656 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-88bd442f-6163-4efa-9414-6ffbad5b63df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:12.991822 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1311/2608] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:12.997037 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72ce8c-8ee7-4b4c-a39c-89d8d444ee82 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:13.010376 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72ce8c-8ee7-4b4c-a39c-89d8d444ee82 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:13.016164 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a553b-2f04-4447-ad7d-c5185f5454ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:13.017020 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a553b-2f04-4447-ad7d-c5185f5454ec admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:13.018442 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a553b-2f04-4447-ad7d-c5185f5454ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:13.018442 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a553b-2f04-4447-ad7d-c5185f5454ec admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:13.018442 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a553b-2f04-4447-ad7d-c5185f5454ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 09:01:13.019153 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a553b-2f04-4447-ad7d-c5185f5454ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:13.020190 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a553b-2f04-4447-ad7d-c5185f5454ec admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:13.042839 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1298/2609] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 09:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 09:01:13.049406 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd442f-6163-4efa-9414-6ffbad5b63df None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:13.059175 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd442f-6163-4efa-9414-6ffbad5b63df None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:13.064228 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244db5b-1dfb-467c-a363-32200b524ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:13.065714 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244db5b-1dfb-467c-a363-32200b524ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:13.065714 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244db5b-1dfb-467c-a363-32200b524ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:13.065949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244db5b-1dfb-467c-a363-32200b524ae7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:13.066511 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8244db5b-1dfb-467c-a363-32200b524ae7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:13.068703 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8244db5b-1dfb-467c-a363-32200b524ae7 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:13.111887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1312/2610] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:01:13 2022] POST /identity/v3/projects => generated 381 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:01:13.117742 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a553b-2f04-4447-ad7d-c5185f5454ec admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:13.133991 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a553b-2f04-4447-ad7d-c5185f5454ec admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:13.191322 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:13.218844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeddca8-ff2e-4e7f-a3fe-d8cbb9762562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:13.219908 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeddca8-ff2e-4e7f-a3fe-d8cbb9762562 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:13.220336 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeddca8-ff2e-4e7f-a3fe-d8cbb9762562 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:13.220800 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeddca8-ff2e-4e7f-a3fe-d8cbb9762562 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:13.221832 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeddca8-ff2e-4e7f-a3fe-d8cbb9762562 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:13.225328 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeddca8-ff2e-4e7f-a3fe-d8cbb9762562 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:13.257656 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1299/2611] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:01:13 2022] POST /identity/v3/users => generated 291 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:01:13.262696 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244db5b-1dfb-467c-a363-32200b524ae7 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:13.281984 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244db5b-1dfb-467c-a363-32200b524ae7 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:13.289338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010b5fd-a7a6-4625-a0bb-0585f89bb509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:13.290150 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010b5fd-a7a6-4625-a0bb-0585f89bb509 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:13.290534 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010b5fd-a7a6-4625-a0bb-0585f89bb509 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:13.290857 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010b5fd-a7a6-4625-a0bb-0585f89bb509 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:13.291779 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4010b5fd-a7a6-4625-a0bb-0585f89bb509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:13.293648 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4010b5fd-a7a6-4625-a0bb-0585f89bb509 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:13.310087 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1313/2612] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:13.310267 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeddca8-ff2e-4e7f-a3fe-d8cbb9762562 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:13.322446 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeddca8-ff2e-4e7f-a3fe-d8cbb9762562 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:13.328194 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef01048-2e5d-4b89-8d4c-56b6e61cc796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:13.328920 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef01048-2e5d-4b89-8d4c-56b6e61cc796 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:13.329267 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef01048-2e5d-4b89-8d4c-56b6e61cc796 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:13.329630 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef01048-2e5d-4b89-8d4c-56b6e61cc796 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:13.330453 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef01048-2e5d-4b89-8d4c-56b6e61cc796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:13.332239 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef01048-2e5d-4b89-8d4c-56b6e61cc796 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:13.350197 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1300/2613] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:13.351696 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010b5fd-a7a6-4625-a0bb-0585f89bb509 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:13.367899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010b5fd-a7a6-4625-a0bb-0585f89bb509 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:13.376148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27550f72-2183-43f2-9a7b-dba86c507acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:13.377010 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27550f72-2183-43f2-9a7b-dba86c507acb admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:13.377390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27550f72-2183-43f2-9a7b-dba86c507acb admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:13.377766 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27550f72-2183-43f2-9a7b-dba86c507acb admin admin] PATH_INFO: `/v3/projects/4b406ced33804e0f848f3daff04263f8/users/c8df656df4c94bd48e7a699457d4e0d2/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:13.400527 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27550f72-2183-43f2-9a7b-dba86c507acb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b406ced33804e0f848f3daff04263f8, user_id=c8df656df4c94bd48e7a699457d4e0d2, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:13.404371 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27550f72-2183-43f2-9a7b-dba86c507acb admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:13.425112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1314/2614] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:01:13 2022] PUT /identity/v3/projects/4b406ced33804e0f848f3daff04263f8/users/c8df656df4c94bd48e7a699457d4e0d2/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:13.436108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c146d38-e908-4f2f-adf3-9d166e535a7a None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:13.436497 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c146d38-e908-4f2f-adf3-9d166e535a7a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:13.436867 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c146d38-e908-4f2f-adf3-9d166e535a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:13.471663 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-1c146d38-e908-4f2f-adf3-9d166e535a7a None None] MFA Rules not processed for user `c8df656df4c94bd48e7a699457d4e0d2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:13.473041 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1c146d38-e908-4f2f-adf3-9d166e535a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:13.551042 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1301/2615] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:13.553271 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27550f72-2183-43f2-9a7b-dba86c507acb admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:13.627631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27550f72-2183-43f2-9a7b-dba86c507acb admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:13.632823 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5707f-f8a3-437c-9051-9e890030c770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:13.633456 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5707f-f8a3-437c-9051-9e890030c770 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:13.633733 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5707f-f8a3-437c-9051-9e890030c770 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:13.634007 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5707f-f8a3-437c-9051-9e890030c770 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:13.634646 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5707f-f8a3-437c-9051-9e890030c770 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:13.636649 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5707f-f8a3-437c-9051-9e890030c770 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:13.672787 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1315/2616] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:01:13 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:01:13.675098 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c146d38-e908-4f2f-adf3-9d166e535a7a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:13.687793 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c146d38-e908-4f2f-adf3-9d166e535a7a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:13.768861 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db90bc3-30d9-4563-855e-2c0c0f444720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:13.769578 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db90bc3-30d9-4563-855e-2c0c0f444720 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:13.769943 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db90bc3-30d9-4563-855e-2c0c0f444720 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:13.770426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db90bc3-30d9-4563-855e-2c0c0f444720 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:13.771155 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db90bc3-30d9-4563-855e-2c0c0f444720 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:13.773599 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db90bc3-30d9-4563-855e-2c0c0f444720 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:13.803586 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1302/2617] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:01:13 2022] POST /identity/v3/users => generated 297 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:01:13.807253 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5707f-f8a3-437c-9051-9e890030c770 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:13.824875 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5707f-f8a3-437c-9051-9e890030c770 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:13.832470 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ba320-afc1-4a35-8c1c-9ad8bf6f7711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:13.833411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ba320-afc1-4a35-8c1c-9ad8bf6f7711 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:13.833860 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ba320-afc1-4a35-8c1c-9ad8bf6f7711 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:13.834281 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ba320-afc1-4a35-8c1c-9ad8bf6f7711 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:13.835858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ba320-afc1-4a35-8c1c-9ad8bf6f7711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:13.837651 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ba320-afc1-4a35-8c1c-9ad8bf6f7711 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:13.851247 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1316/2618] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:13.855705 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db90bc3-30d9-4563-855e-2c0c0f444720 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:13.869995 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db90bc3-30d9-4563-855e-2c0c0f444720 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:13.878356 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b25109-01d7-4b1e-a8d8-8aa0b49453cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:13.883172 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b25109-01d7-4b1e-a8d8-8aa0b49453cf admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:13.883172 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b25109-01d7-4b1e-a8d8-8aa0b49453cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:13.883172 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b25109-01d7-4b1e-a8d8-8aa0b49453cf admin admin] PATH_INFO: `/v3/projects/533192ea55514b0580ca35661c7f89f2/users/05793becbed74c4e9e58de657f6d483e/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:13.899325 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b25109-01d7-4b1e-a8d8-8aa0b49453cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=533192ea55514b0580ca35661c7f89f2, user_id=05793becbed74c4e9e58de657f6d483e, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:13.902241 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b25109-01d7-4b1e-a8d8-8aa0b49453cf admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:13.918093 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1303/2619] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:01:13 2022] PUT /identity/v3/projects/533192ea55514b0580ca35661c7f89f2/users/05793becbed74c4e9e58de657f6d483e/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 09:01:13.923892 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ba320-afc1-4a35-8c1c-9ad8bf6f7711 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:13.941512 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ba320-afc1-4a35-8c1c-9ad8bf6f7711 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:14.017084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c53f77-b655-4c68-a785-491e5a121446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:14.017801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c53f77-b655-4c68-a785-491e5a121446 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:14.018082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c53f77-b655-4c68-a785-491e5a121446 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:14.018358 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c53f77-b655-4c68-a785-491e5a121446 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:14.019084 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c53f77-b655-4c68-a785-491e5a121446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:14.020692 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c53f77-b655-4c68-a785-491e5a121446 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:14.027259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:14.042911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1317/2620] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:14.044380 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b25109-01d7-4b1e-a8d8-8aa0b49453cf admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:14.056351 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b25109-01d7-4b1e-a8d8-8aa0b49453cf admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:14.062671 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d10deeb-4bb3-40e5-8362-3dc939ff0b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:14.063421 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10deeb-4bb3-40e5-8362-3dc939ff0b76 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:14.064264 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10deeb-4bb3-40e5-8362-3dc939ff0b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:14.064264 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10deeb-4bb3-40e5-8362-3dc939ff0b76 admin admin] PATH_INFO: `/v3/projects/533192ea55514b0580ca35661c7f89f2/users/05793becbed74c4e9e58de657f6d483e/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:14.066775 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d10deeb-4bb3-40e5-8362-3dc939ff0b76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=533192ea55514b0580ca35661c7f89f2, user_id=05793becbed74c4e9e58de657f6d483e, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:14.069475 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d10deeb-4bb3-40e5-8362-3dc939ff0b76 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:14.076488 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:14.083965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1304/2621] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:01:14 2022] PUT /identity/v3/projects/533192ea55514b0580ca35661c7f89f2/users/05793becbed74c4e9e58de657f6d483e/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:14.099303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e83ae2-4e26-4bb7-8487-a69da54242ae None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:14.099586 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e83ae2-4e26-4bb7-8487-a69da54242ae None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:14.099858 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e83ae2-4e26-4bb7-8487-a69da54242ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:14.131729 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-25e83ae2-4e26-4bb7-8487-a69da54242ae None None] MFA Rules not processed for user `05793becbed74c4e9e58de657f6d483e`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:14.132892 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-25e83ae2-4e26-4bb7-8487-a69da54242ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:14.207156 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1318/2622] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:14 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:14.220028 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d10deeb-4bb3-40e5-8362-3dc939ff0b76 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:14.222058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6d10deeb-4bb3-40e5-8362-3dc939ff0b76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:14.292923 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d10deeb-4bb3-40e5-8362-3dc939ff0b76 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:14.297005 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b8e37-5312-4431-af55-8d161d091772 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:14.297718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b8e37-5312-4431-af55-8d161d091772 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:14.297921 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b8e37-5312-4431-af55-8d161d091772 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:14.298404 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b8e37-5312-4431-af55-8d161d091772 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:14.308930 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b8e37-5312-4431-af55-8d161d091772 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:14.310389 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b8e37-5312-4431-af55-8d161d091772 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:14.321480 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1305/2623] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:01:14 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:01:14.409995 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e83ae2-4e26-4bb7-8487-a69da54242ae None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:14.421621 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e83ae2-4e26-4bb7-8487-a69da54242ae None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:14.426618 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768c1f9-2845-4949-b9c5-14c617e84ec0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:14.427292 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768c1f9-2845-4949-b9c5-14c617e84ec0 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:14.427599 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768c1f9-2845-4949-b9c5-14c617e84ec0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:14.427866 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768c1f9-2845-4949-b9c5-14c617e84ec0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:14.489587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768c1f9-2845-4949-b9c5-14c617e84ec0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:14.491096 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768c1f9-2845-4949-b9c5-14c617e84ec0 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:14.500446 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1319/2624] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:01:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:01:24.315917 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=26 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:24.496437 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=62 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:24.918428 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b8e37-5312-4431-af55-8d161d091772 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:24.994542 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b8e37-5312-4431-af55-8d161d091772 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:24.999964 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7d99c-1dc6-40f4-a3e7-89e4bdf7568b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:25.000604 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7d99c-1dc6-40f4-a3e7-89e4bdf7568b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:25.000971 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7d99c-1dc6-40f4-a3e7-89e4bdf7568b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:25.001234 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d7d99c-1dc6-40f4-a3e7-89e4bdf7568b admin admin] PATH_INFO: `/v3/users/c8df656df4c94bd48e7a699457d4e0d2` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:25.002527 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7d99c-1dc6-40f4-a3e7-89e4bdf7568b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8df656df4c94bd48e7a699457d4e0d2)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:25.004517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d7d99c-1dc6-40f4-a3e7-89e4bdf7568b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:25.063719 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-67d7d99c-1dc6-40f4-a3e7-89e4bdf7568b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8df656df4c94bd48e7a699457d4e0d2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:25.074943 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-67d7d99c-1dc6-40f4-a3e7-89e4bdf7568b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8df656df4c94bd48e7a699457d4e0d2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:25.087091 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-67d7d99c-1dc6-40f4-a3e7-89e4bdf7568b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8df656df4c94bd48e7a699457d4e0d2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:25.087839 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-67d7d99c-1dc6-40f4-a3e7-89e4bdf7568b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8df656df4c94bd48e7a699457d4e0d2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:25.088524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-67d7d99c-1dc6-40f4-a3e7-89e4bdf7568b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8df656df4c94bd48e7a699457d4e0d2'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:25.110747 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1306/2625] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:01:24 2022] DELETE /identity/v3/users/c8df656df4c94bd48e7a699457d4e0d2 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:25.116088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01762ed5-4f81-4284-aca8-3ec1be93cb6d None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:25.116441 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01762ed5-4f81-4284-aca8-3ec1be93cb6d None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:25.116810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01762ed5-4f81-4284-aca8-3ec1be93cb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:25.139308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-01762ed5-4f81-4284-aca8-3ec1be93cb6d None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:25.141214 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-01762ed5-4f81-4284-aca8-3ec1be93cb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:25.248087 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1320/2626] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:25.263939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7d99c-1dc6-40f4-a3e7-89e4bdf7568b admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:25.265673 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-67d7d99c-1dc6-40f4-a3e7-89e4bdf7568b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:25.355744 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d7d99c-1dc6-40f4-a3e7-89e4bdf7568b admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:25.363494 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a341ab-bb3b-4c27-9a2c-2349e094895c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:25.364283 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a341ab-bb3b-4c27-9a2c-2349e094895c service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:25.364572 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a341ab-bb3b-4c27-9a2c-2349e094895c service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:25.364829 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a341ab-bb3b-4c27-9a2c-2349e094895c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:25.381414 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a341ab-bb3b-4c27-9a2c-2349e094895c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:25.383844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a341ab-bb3b-4c27-9a2c-2349e094895c service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:25.395045 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1307/2627] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:01:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 09:01:25.786855 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01762ed5-4f81-4284-aca8-3ec1be93cb6d None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:25.789228 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-01762ed5-4f81-4284-aca8-3ec1be93cb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:25.821504 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01762ed5-4f81-4284-aca8-3ec1be93cb6d None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:25.831317 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079dba18-8065-4fb5-8a3c-08a35c74d9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:25.832454 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dba18-8065-4fb5-8a3c-08a35c74d9e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:25.832864 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dba18-8065-4fb5-8a3c-08a35c74d9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:25.833269 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dba18-8065-4fb5-8a3c-08a35c74d9e3 admin admin] PATH_INFO: `/v3/projects/4b406ced33804e0f848f3daff04263f8` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:25.834672 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079dba18-8065-4fb5-8a3c-08a35c74d9e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b406ced33804e0f848f3daff04263f8)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:25.838206 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079dba18-8065-4fb5-8a3c-08a35c74d9e3 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:25.893022 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-079dba18-8065-4fb5-8a3c-08a35c74d9e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b406ced33804e0f848f3daff04263f8', 'request_id': 'req-079dba18-8065-4fb5-8a3c-08a35c74d9e3'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:25.898901 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-079dba18-8065-4fb5-8a3c-08a35c74d9e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b406ced33804e0f848f3daff04263f8', 'request_id': 'req-079dba18-8065-4fb5-8a3c-08a35c74d9e3'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:25.910583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-079dba18-8065-4fb5-8a3c-08a35c74d9e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b406ced33804e0f848f3daff04263f8', 'request_id': 'req-079dba18-8065-4fb5-8a3c-08a35c74d9e3'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:25.924307 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-079dba18-8065-4fb5-8a3c-08a35c74d9e3 admin admin] The token cache is being invalidate because project 4b406ced33804e0f848f3daff04263f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:01:25.924645 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-079dba18-8065-4fb5-8a3c-08a35c74d9e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:25.925420 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-079dba18-8065-4fb5-8a3c-08a35c74d9e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:25.927405 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1321/2628] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:01:25 2022] DELETE /identity/v3/projects/4b406ced33804e0f848f3daff04263f8 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:25.931804 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a341ab-bb3b-4c27-9a2c-2349e094895c service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:25.942159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a341ab-bb3b-4c27-9a2c-2349e094895c service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:25.965568 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:26.008292 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9707f-e83c-46bd-a98f-1974b7d5e446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:26.009013 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9707f-e83c-46bd-a98f-1974b7d5e446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:26.009289 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9707f-e83c-46bd-a98f-1974b7d5e446 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:26.009664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9707f-e83c-46bd-a98f-1974b7d5e446 admin admin] PATH_INFO: `/v3/users/05793becbed74c4e9e58de657f6d483e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:26.010978 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9707f-e83c-46bd-a98f-1974b7d5e446 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05793becbed74c4e9e58de657f6d483e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:26.012912 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9707f-e83c-46bd-a98f-1974b7d5e446 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:26.064969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-f2d9707f-e83c-46bd-a98f-1974b7d5e446 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05793becbed74c4e9e58de657f6d483e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:26.074126 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-f2d9707f-e83c-46bd-a98f-1974b7d5e446 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05793becbed74c4e9e58de657f6d483e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:26.083451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-f2d9707f-e83c-46bd-a98f-1974b7d5e446 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05793becbed74c4e9e58de657f6d483e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:26.084205 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-f2d9707f-e83c-46bd-a98f-1974b7d5e446 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05793becbed74c4e9e58de657f6d483e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:26.084974 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-f2d9707f-e83c-46bd-a98f-1974b7d5e446 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05793becbed74c4e9e58de657f6d483e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:26.104120 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1308/2629] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:01:25 2022] DELETE /identity/v3/users/05793becbed74c4e9e58de657f6d483e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:26.434373 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079dba18-8065-4fb5-8a3c-08a35c74d9e3 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:26.437429 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-079dba18-8065-4fb5-8a3c-08a35c74d9e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:26.456324 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079dba18-8065-4fb5-8a3c-08a35c74d9e3 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:26.556064 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f46417-9c3e-4da7-8bd5-676bc8ae26ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:26.556643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f46417-9c3e-4da7-8bd5-676bc8ae26ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:26.556941 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f46417-9c3e-4da7-8bd5-676bc8ae26ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:26.557214 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f46417-9c3e-4da7-8bd5-676bc8ae26ac admin admin] PATH_INFO: `/v3/projects/533192ea55514b0580ca35661c7f89f2` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:26.558421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f46417-9c3e-4da7-8bd5-676bc8ae26ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=533192ea55514b0580ca35661c7f89f2)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:26.560458 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f46417-9c3e-4da7-8bd5-676bc8ae26ac admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:26.595953 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:26.621318 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d4f46417-9c3e-4da7-8bd5-676bc8ae26ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '533192ea55514b0580ca35661c7f89f2', 'request_id': 'req-d4f46417-9c3e-4da7-8bd5-676bc8ae26ac'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:26.627515 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d4f46417-9c3e-4da7-8bd5-676bc8ae26ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '533192ea55514b0580ca35661c7f89f2', 'request_id': 'req-d4f46417-9c3e-4da7-8bd5-676bc8ae26ac'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:26.635761 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d4f46417-9c3e-4da7-8bd5-676bc8ae26ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '533192ea55514b0580ca35661c7f89f2', 'request_id': 'req-d4f46417-9c3e-4da7-8bd5-676bc8ae26ac'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:26.648698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d4f46417-9c3e-4da7-8bd5-676bc8ae26ac admin admin] The token cache is being invalidate because project 533192ea55514b0580ca35661c7f89f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:01:26.648987 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d4f46417-9c3e-4da7-8bd5-676bc8ae26ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:26.649669 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-d4f46417-9c3e-4da7-8bd5-676bc8ae26ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:26.651247 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1322/2630] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:01:26 2022] DELETE /identity/v3/projects/533192ea55514b0580ca35661c7f89f2 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:26.656362 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28278b10-37b6-4894-902c-8a91808d9a2f None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:26.656863 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28278b10-37b6-4894-902c-8a91808d9a2f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:26.656968 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28278b10-37b6-4894-902c-8a91808d9a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:26.679840 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-28278b10-37b6-4894-902c-8a91808d9a2f None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:26.681171 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-28278b10-37b6-4894-902c-8a91808d9a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:26.794500 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1309/2631] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 09:01:26.798940 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f0c4c-8a53-42bd-b7ba-39475c81daec None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:26.798940 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f0c4c-8a53-42bd-b7ba-39475c81daec None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:26.798940 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f0c4c-8a53-42bd-b7ba-39475c81daec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:26.822790 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-fd0f0c4c-8a53-42bd-b7ba-39475c81daec None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:26.824039 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-fd0f0c4c-8a53-42bd-b7ba-39475c81daec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:26.835824 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1323/2632] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:26.840426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28278b10-37b6-4894-902c-8a91808d9a2f None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:26.852331 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28278b10-37b6-4894-902c-8a91808d9a2f None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:26.859917 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426cbcdd-2a5a-4e26-922f-d0642db94267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:26.860680 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cbcdd-2a5a-4e26-922f-d0642db94267 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:26.861140 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cbcdd-2a5a-4e26-922f-d0642db94267 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:26.861385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cbcdd-2a5a-4e26-922f-d0642db94267 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:26.862300 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426cbcdd-2a5a-4e26-922f-d0642db94267 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 09:01:26.862706 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426cbcdd-2a5a-4e26-922f-d0642db94267 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:26.864573 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426cbcdd-2a5a-4e26-922f-d0642db94267 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:26.879947 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1310/2633] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 09:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 09:01:26.886150 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f0c4c-8a53-42bd-b7ba-39475c81daec None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:26.906755 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f0c4c-8a53-42bd-b7ba-39475c81daec None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:26.915299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbe729-8970-409c-befd-fac307a69a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:26.915299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbe729-8970-409c-befd-fac307a69a5a admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:26.915299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbe729-8970-409c-befd-fac307a69a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:26.915299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbe729-8970-409c-befd-fac307a69a5a admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:26.915950 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dbe729-8970-409c-befd-fac307a69a5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:26.916212 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46931901-9ae7-49a4-a0b9-5c86eb5cac3a None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:26.916473 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46931901-9ae7-49a4-a0b9-5c86eb5cac3a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:26.916701 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dbe729-8970-409c-befd-fac307a69a5a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:26.916847 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46931901-9ae7-49a4-a0b9-5c86eb5cac3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:26.937828 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-46931901-9ae7-49a4-a0b9-5c86eb5cac3a None None] MFA Rules not processed for user `34eaab912a91496daeedbaf63f5ddcbb`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:26.939089 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-46931901-9ae7-49a4-a0b9-5c86eb5cac3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:26.976075 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1324/2634] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:01:26 2022] POST /identity/v3/projects => generated 387 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:01:26.976372 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:26.977309 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbe729-8970-409c-befd-fac307a69a5a admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:27.014206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1311/2635] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:27.057825 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbe729-8970-409c-befd-fac307a69a5a admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:27.063994 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164e123-36f0-4d95-a9f4-0f963242a705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:27.064541 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164e123-36f0-4d95-a9f4-0f963242a705 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:27.064831 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164e123-36f0-4d95-a9f4-0f963242a705 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:27.065092 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164e123-36f0-4d95-a9f4-0f963242a705 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:27.065726 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2164e123-36f0-4d95-a9f4-0f963242a705 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:27.068771 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2164e123-36f0-4d95-a9f4-0f963242a705 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:27.097641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1325/2636] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:01:26 2022] POST /identity/v3/users => generated 294 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:01:27.102494 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46931901-9ae7-49a4-a0b9-5c86eb5cac3a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:27.115551 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46931901-9ae7-49a4-a0b9-5c86eb5cac3a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:27.120664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19828b-2374-4867-9a43-7f080bb910af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:27.121455 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19828b-2374-4867-9a43-7f080bb910af admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:27.121748 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19828b-2374-4867-9a43-7f080bb910af admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:27.122008 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19828b-2374-4867-9a43-7f080bb910af admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:27.122660 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d19828b-2374-4867-9a43-7f080bb910af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:27.124294 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d19828b-2374-4867-9a43-7f080bb910af admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:27.135772 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1312/2637] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:27.140459 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164e123-36f0-4d95-a9f4-0f963242a705 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:27.154792 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164e123-36f0-4d95-a9f4-0f963242a705 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:27.161830 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb4d61-fb77-48df-94d6-8261b8a59422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:27.162449 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb4d61-fb77-48df-94d6-8261b8a59422 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:27.162725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb4d61-fb77-48df-94d6-8261b8a59422 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:27.163028 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb4d61-fb77-48df-94d6-8261b8a59422 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:27.163682 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bb4d61-fb77-48df-94d6-8261b8a59422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:27.165315 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bb4d61-fb77-48df-94d6-8261b8a59422 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:27.176430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1326/2638] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:27.182588 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19828b-2374-4867-9a43-7f080bb910af admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:27.205378 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d19828b-2374-4867-9a43-7f080bb910af admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:27.214049 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e259a-7a74-4c75-883a-974ef72a5478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:27.215223 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e259a-7a74-4c75-883a-974ef72a5478 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:27.215725 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e259a-7a74-4c75-883a-974ef72a5478 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:27.216222 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e259a-7a74-4c75-883a-974ef72a5478 admin admin] PATH_INFO: `/v3/projects/0c1b50e890d44fe7ab875825f9434a5a/users/29b9861aec3b420abe00f484ac8abf92/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:27.245845 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e259a-7a74-4c75-883a-974ef72a5478 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c1b50e890d44fe7ab875825f9434a5a, user_id=29b9861aec3b420abe00f484ac8abf92, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:27.248685 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e259a-7a74-4c75-883a-974ef72a5478 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:27.266282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1313/2639] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:01:27 2022] PUT /identity/v3/projects/0c1b50e890d44fe7ab875825f9434a5a/users/29b9861aec3b420abe00f484ac8abf92/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:27.279145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11ead2-0f8b-4413-98ea-4db2d8c0dab8 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:27.279401 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11ead2-0f8b-4413-98ea-4db2d8c0dab8 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:27.279714 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11ead2-0f8b-4413-98ea-4db2d8c0dab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:27.325258 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-fa11ead2-0f8b-4413-98ea-4db2d8c0dab8 None None] MFA Rules not processed for user `29b9861aec3b420abe00f484ac8abf92`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:27.326693 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-fa11ead2-0f8b-4413-98ea-4db2d8c0dab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:27.388471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:27.420979 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1327/2640] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:27 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:27.424789 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e259a-7a74-4c75-883a-974ef72a5478 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:27.498839 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e259a-7a74-4c75-883a-974ef72a5478 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:27.504032 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb6712-1d63-4163-bda5-eba72acc8f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:27.504647 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb6712-1d63-4163-bda5-eba72acc8f73 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:27.504922 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb6712-1d63-4163-bda5-eba72acc8f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:27.505825 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb6712-1d63-4163-bda5-eba72acc8f73 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:27.506535 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb6712-1d63-4163-bda5-eba72acc8f73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:27.508563 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb6712-1d63-4163-bda5-eba72acc8f73 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:27.539220 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1314/2641] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:01:27 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:01:27.543675 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11ead2-0f8b-4413-98ea-4db2d8c0dab8 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:27.555773 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11ead2-0f8b-4413-98ea-4db2d8c0dab8 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:27.621003 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7b557-515d-42d4-9305-af9eb91782f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:27.622218 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7b557-515d-42d4-9305-af9eb91782f7 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:27.622218 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7b557-515d-42d4-9305-af9eb91782f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:27.622218 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7b557-515d-42d4-9305-af9eb91782f7 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:27.622921 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d7b557-515d-42d4-9305-af9eb91782f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:27.625076 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d7b557-515d-42d4-9305-af9eb91782f7 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:27.660653 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1328/2642] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:01:27 2022] POST /identity/v3/users => generated 299 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:01:27.666582 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb6712-1d63-4163-bda5-eba72acc8f73 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:27.682468 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb6712-1d63-4163-bda5-eba72acc8f73 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:27.689374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86ed40-98da-42b2-8a8a-b819aeae2a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:27.690493 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86ed40-98da-42b2-8a8a-b819aeae2a9d admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:27.690974 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86ed40-98da-42b2-8a8a-b819aeae2a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:27.691466 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86ed40-98da-42b2-8a8a-b819aeae2a9d admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:27.692164 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c86ed40-98da-42b2-8a8a-b819aeae2a9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:27.693847 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c86ed40-98da-42b2-8a8a-b819aeae2a9d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:27.706797 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1315/2643] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:27.713049 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7b557-515d-42d4-9305-af9eb91782f7 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:27.725789 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7b557-515d-42d4-9305-af9eb91782f7 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:27.731039 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d161ab-549f-466a-86d4-993bc71938c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:27.731769 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d161ab-549f-466a-86d4-993bc71938c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:27.732206 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d161ab-549f-466a-86d4-993bc71938c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:27.732498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d161ab-549f-466a-86d4-993bc71938c5 admin admin] PATH_INFO: `/v3/projects/1264990108a144d1916bcaf36d81c964/users/555d9e541ec74c43b5d796297f54298c/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:27.759490 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d161ab-549f-466a-86d4-993bc71938c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1264990108a144d1916bcaf36d81c964, user_id=555d9e541ec74c43b5d796297f54298c, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:27.762311 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d161ab-549f-466a-86d4-993bc71938c5 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:27.782615 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1329/2644] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:01:27 2022] PUT /identity/v3/projects/1264990108a144d1916bcaf36d81c964/users/555d9e541ec74c43b5d796297f54298c/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:27.787968 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86ed40-98da-42b2-8a8a-b819aeae2a9d admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:27.800430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86ed40-98da-42b2-8a8a-b819aeae2a9d admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:27.877371 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d333ef87-5de2-4021-9e2b-0debd5553a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:27.878241 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333ef87-5de2-4021-9e2b-0debd5553a05 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:27.878775 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333ef87-5de2-4021-9e2b-0debd5553a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:27.879281 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333ef87-5de2-4021-9e2b-0debd5553a05 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:27.880110 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d333ef87-5de2-4021-9e2b-0debd5553a05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:27.882737 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d333ef87-5de2-4021-9e2b-0debd5553a05 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:27.899902 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1316/2645] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 09:01:27.905150 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d161ab-549f-466a-86d4-993bc71938c5 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:27.934799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d161ab-549f-466a-86d4-993bc71938c5 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:27.945414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d5fa0-e6d9-4cdf-97bd-1108d92d6543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:27.947520 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d5fa0-e6d9-4cdf-97bd-1108d92d6543 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:27.947520 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d5fa0-e6d9-4cdf-97bd-1108d92d6543 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:27.949379 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d5fa0-e6d9-4cdf-97bd-1108d92d6543 admin admin] PATH_INFO: `/v3/projects/1264990108a144d1916bcaf36d81c964/users/555d9e541ec74c43b5d796297f54298c/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:27.953402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d5fa0-e6d9-4cdf-97bd-1108d92d6543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1264990108a144d1916bcaf36d81c964, user_id=555d9e541ec74c43b5d796297f54298c, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:27.958984 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207d5fa0-e6d9-4cdf-97bd-1108d92d6543 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:27.980848 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1330/2646] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:01:27 2022] PUT /identity/v3/projects/1264990108a144d1916bcaf36d81c964/users/555d9e541ec74c43b5d796297f54298c/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:27.994876 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3adaa34-1d72-496d-bf23-4f5b8107ef37 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:27.995906 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3adaa34-1d72-496d-bf23-4f5b8107ef37 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:27.996431 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3adaa34-1d72-496d-bf23-4f5b8107ef37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:28.033997 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:28.077416 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-a3adaa34-1d72-496d-bf23-4f5b8107ef37 None None] MFA Rules not processed for user `555d9e541ec74c43b5d796297f54298c`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:28.078695 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a3adaa34-1d72-496d-bf23-4f5b8107ef37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:28.197211 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1317/2647] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:27 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 09:01:28.211332 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d5fa0-e6d9-4cdf-97bd-1108d92d6543 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:28.213464 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-207d5fa0-e6d9-4cdf-97bd-1108d92d6543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:28.312570 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207d5fa0-e6d9-4cdf-97bd-1108d92d6543 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:28.318799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeaa4cc-6c1c-4971-b19f-bcc26760da8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:28.320442 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeaa4cc-6c1c-4971-b19f-bcc26760da8b service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:28.320442 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeaa4cc-6c1c-4971-b19f-bcc26760da8b service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:28.320442 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeaa4cc-6c1c-4971-b19f-bcc26760da8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:28.339739 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeaa4cc-6c1c-4971-b19f-bcc26760da8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:28.341580 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeaa4cc-6c1c-4971-b19f-bcc26760da8b service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:28.353926 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1331/2648] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:01:28 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:01:28.451805 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3adaa34-1d72-496d-bf23-4f5b8107ef37 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:28.549419 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3adaa34-1d72-496d-bf23-4f5b8107ef37 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:28.557558 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e7730-6694-4054-a807-392785a4711e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:28.558609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e7730-6694-4054-a807-392785a4711e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:28.559297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e7730-6694-4054-a807-392785a4711e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:28.559851 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e7730-6694-4054-a807-392785a4711e admin admin] PATH_INFO: `/v3/users/29b9861aec3b420abe00f484ac8abf92` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:28.562569 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e7730-6694-4054-a807-392785a4711e admin admin] RBAC: Authorizing `identity:delete_user(user_id=29b9861aec3b420abe00f484ac8abf92)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:28.565208 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e7730-6694-4054-a807-392785a4711e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:28.649836 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-352e7730-6694-4054-a807-392785a4711e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29b9861aec3b420abe00f484ac8abf92'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:28.658671 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-352e7730-6694-4054-a807-392785a4711e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29b9861aec3b420abe00f484ac8abf92'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:28.667083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-352e7730-6694-4054-a807-392785a4711e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29b9861aec3b420abe00f484ac8abf92'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:28.667940 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-352e7730-6694-4054-a807-392785a4711e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29b9861aec3b420abe00f484ac8abf92'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:28.668657 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-352e7730-6694-4054-a807-392785a4711e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29b9861aec3b420abe00f484ac8abf92'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:28.695654 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1318/2649] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:01:28 2022] DELETE /identity/v3/users/29b9861aec3b420abe00f484ac8abf92 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:28.699997 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6daa3-c7e5-4c6f-a248-bb5869b25541 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:28.700263 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6daa3-c7e5-4c6f-a248-bb5869b25541 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:28.700542 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6daa3-c7e5-4c6f-a248-bb5869b25541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:28.724895 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-62e6daa3-c7e5-4c6f-a248-bb5869b25541 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:28.725420 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-62e6daa3-c7e5-4c6f-a248-bb5869b25541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:28.780190 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:28.830408 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1332/2650] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:28.848259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e7730-6694-4054-a807-392785a4711e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:28.851774 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-352e7730-6694-4054-a807-392785a4711e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:28.874402 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:28.974541 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e7730-6694-4054-a807-392785a4711e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:28.981754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d929c22a-d05c-4edc-a815-6fa6a5896997 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:28.982760 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929c22a-d05c-4edc-a815-6fa6a5896997 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:28.983299 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929c22a-d05c-4edc-a815-6fa6a5896997 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:28.983719 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929c22a-d05c-4edc-a815-6fa6a5896997 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:29.007898 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d929c22a-d05c-4edc-a815-6fa6a5896997 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:29.010609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d929c22a-d05c-4edc-a815-6fa6a5896997 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:29.024259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1319/2651] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:01:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:01:29.667968 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e6daa3-c7e5-4c6f-a248-bb5869b25541 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:29.669683 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-62e6daa3-c7e5-4c6f-a248-bb5869b25541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:29.690851 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e6daa3-c7e5-4c6f-a248-bb5869b25541 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:29.707297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0347f4f-73fd-45ae-bc20-f3850cbdb0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:29.707972 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0347f4f-73fd-45ae-bc20-f3850cbdb0c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:29.708259 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0347f4f-73fd-45ae-bc20-f3850cbdb0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:29.708529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0347f4f-73fd-45ae-bc20-f3850cbdb0c4 admin admin] PATH_INFO: `/v3/projects/0c1b50e890d44fe7ab875825f9434a5a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:29.713661 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0347f4f-73fd-45ae-bc20-f3850cbdb0c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c1b50e890d44fe7ab875825f9434a5a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:29.715934 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0347f4f-73fd-45ae-bc20-f3850cbdb0c4 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:29.845664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c0347f4f-73fd-45ae-bc20-f3850cbdb0c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c1b50e890d44fe7ab875825f9434a5a', 'request_id': 'req-c0347f4f-73fd-45ae-bc20-f3850cbdb0c4'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:29.857719 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c0347f4f-73fd-45ae-bc20-f3850cbdb0c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c1b50e890d44fe7ab875825f9434a5a', 'request_id': 'req-c0347f4f-73fd-45ae-bc20-f3850cbdb0c4'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:29.875878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c0347f4f-73fd-45ae-bc20-f3850cbdb0c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c1b50e890d44fe7ab875825f9434a5a', 'request_id': 'req-c0347f4f-73fd-45ae-bc20-f3850cbdb0c4'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:29.902614 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c0347f4f-73fd-45ae-bc20-f3850cbdb0c4 admin admin] The token cache is being invalidate because project 0c1b50e890d44fe7ab875825f9434a5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:01:29.902976 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c0347f4f-73fd-45ae-bc20-f3850cbdb0c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:29.903780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c0347f4f-73fd-45ae-bc20-f3850cbdb0c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:29.905362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1333/2652] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:01:29 2022] DELETE /identity/v3/projects/0c1b50e890d44fe7ab875825f9434a5a => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:29.909748 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d929c22a-d05c-4edc-a815-6fa6a5896997 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:29.921719 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d929c22a-d05c-4edc-a815-6fa6a5896997 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:29.997198 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e7e2c6-1a60-4bac-9ee3-387caf4e4ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:29.997873 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7e2c6-1a60-4bac-9ee3-387caf4e4ead admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:29.998155 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7e2c6-1a60-4bac-9ee3-387caf4e4ead admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:29.998430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7e2c6-1a60-4bac-9ee3-387caf4e4ead admin admin] PATH_INFO: `/v3/users/555d9e541ec74c43b5d796297f54298c` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:29.999937 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e7e2c6-1a60-4bac-9ee3-387caf4e4ead admin admin] RBAC: Authorizing `identity:delete_user(user_id=555d9e541ec74c43b5d796297f54298c)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:30.002011 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e7e2c6-1a60-4bac-9ee3-387caf4e4ead admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:30.077149 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-62e7e2c6-1a60-4bac-9ee3-387caf4e4ead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '555d9e541ec74c43b5d796297f54298c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:30.092509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-62e7e2c6-1a60-4bac-9ee3-387caf4e4ead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '555d9e541ec74c43b5d796297f54298c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:30.106114 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-62e7e2c6-1a60-4bac-9ee3-387caf4e4ead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '555d9e541ec74c43b5d796297f54298c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:30.107160 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-62e7e2c6-1a60-4bac-9ee3-387caf4e4ead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '555d9e541ec74c43b5d796297f54298c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:30.107961 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-62e7e2c6-1a60-4bac-9ee3-387caf4e4ead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '555d9e541ec74c43b5d796297f54298c'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:30.140957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1320/2653] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:01:29 2022] DELETE /identity/v3/users/555d9e541ec74c43b5d796297f54298c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:30.992160 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0347f4f-73fd-45ae-bc20-f3850cbdb0c4 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:30.994644 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c0347f4f-73fd-45ae-bc20-f3850cbdb0c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:31.012737 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0347f4f-73fd-45ae-bc20-f3850cbdb0c4 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:31.160324 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8240e33f-bbe3-47fa-8862-429d70d95761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:31.161167 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240e33f-bbe3-47fa-8862-429d70d95761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:31.161608 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240e33f-bbe3-47fa-8862-429d70d95761 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:31.161948 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240e33f-bbe3-47fa-8862-429d70d95761 admin admin] PATH_INFO: `/v3/projects/1264990108a144d1916bcaf36d81c964` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:31.176673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8240e33f-bbe3-47fa-8862-429d70d95761 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1264990108a144d1916bcaf36d81c964)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:31.183991 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8240e33f-bbe3-47fa-8862-429d70d95761 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:31.275251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8240e33f-bbe3-47fa-8862-429d70d95761 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1264990108a144d1916bcaf36d81c964', 'request_id': 'req-8240e33f-bbe3-47fa-8862-429d70d95761'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:31.292498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8240e33f-bbe3-47fa-8862-429d70d95761 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1264990108a144d1916bcaf36d81c964', 'request_id': 'req-8240e33f-bbe3-47fa-8862-429d70d95761'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:31.312243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8240e33f-bbe3-47fa-8862-429d70d95761 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1264990108a144d1916bcaf36d81c964', 'request_id': 'req-8240e33f-bbe3-47fa-8862-429d70d95761'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:31.313828 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:31.347293 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8240e33f-bbe3-47fa-8862-429d70d95761 admin admin] The token cache is being invalidate because project 1264990108a144d1916bcaf36d81c964 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:01:31.348082 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8240e33f-bbe3-47fa-8862-429d70d95761 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:31.349025 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-8240e33f-bbe3-47fa-8862-429d70d95761 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:31.350832 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1334/2654] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:01:30 2022] DELETE /identity/v3/projects/1264990108a144d1916bcaf36d81c964 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:31.356477 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f9d59-e694-4715-9fbb-c694d8f6f053 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:31.357516 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f9d59-e694-4715-9fbb-c694d8f6f053 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:31.357516 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f9d59-e694-4715-9fbb-c694d8f6f053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:31.393541 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-fc4f9d59-e694-4715-9fbb-c694d8f6f053 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:31.396190 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-fc4f9d59-e694-4715-9fbb-c694d8f6f053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:31.452672 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:31.519932 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1321/2655] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:31.528434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14636ed9-fb99-4e71-bce4-3596a9220832 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:31.528933 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14636ed9-fb99-4e71-bce4-3596a9220832 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:31.529281 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14636ed9-fb99-4e71-bce4-3596a9220832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:31.568412 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-14636ed9-fb99-4e71-bce4-3596a9220832 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:31.569730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-14636ed9-fb99-4e71-bce4-3596a9220832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:31.578270 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1335/2656] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 09:01:31.583967 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f9d59-e694-4715-9fbb-c694d8f6f053 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:31.596595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f9d59-e694-4715-9fbb-c694d8f6f053 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:31.603257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521c960-2a0b-40af-a3c6-42f5fe6b2fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:31.604485 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521c960-2a0b-40af-a3c6-42f5fe6b2fee admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:31.604850 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521c960-2a0b-40af-a3c6-42f5fe6b2fee admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:31.605206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521c960-2a0b-40af-a3c6-42f5fe6b2fee admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:31.605976 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6521c960-2a0b-40af-a3c6-42f5fe6b2fee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 09:01:31.606530 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6521c960-2a0b-40af-a3c6-42f5fe6b2fee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:31.608335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6521c960-2a0b-40af-a3c6-42f5fe6b2fee admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:31.625754 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1322/2657] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 09:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 09:01:31.631203 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14636ed9-fb99-4e71-bce4-3596a9220832 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:31.642878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14636ed9-fb99-4e71-bce4-3596a9220832 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:31.648909 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d646fc1-5a6b-46e8-a895-0ed9cf17d594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:31.649909 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d646fc1-5a6b-46e8-a895-0ed9cf17d594 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:31.650200 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d646fc1-5a6b-46e8-a895-0ed9cf17d594 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:31.650479 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d646fc1-5a6b-46e8-a895-0ed9cf17d594 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:31.651199 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d646fc1-5a6b-46e8-a895-0ed9cf17d594 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:31.653212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d646fc1-5a6b-46e8-a895-0ed9cf17d594 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:31.696800 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1336/2658] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:01:31 2022] POST /identity/v3/projects => generated 379 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:01:31.698077 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521c960-2a0b-40af-a3c6-42f5fe6b2fee admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:31.713899 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521c960-2a0b-40af-a3c6-42f5fe6b2fee admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:31.832753 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d1c12-dc11-4db2-a516-2ec6f542bcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:31.833540 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d1c12-dc11-4db2-a516-2ec6f542bcb5 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:31.833834 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d1c12-dc11-4db2-a516-2ec6f542bcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:31.834121 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d1c12-dc11-4db2-a516-2ec6f542bcb5 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:31.834817 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d1c12-dc11-4db2-a516-2ec6f542bcb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:31.842698 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d1c12-dc11-4db2-a516-2ec6f542bcb5 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:31.883193 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1323/2659] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:01:31 2022] POST /identity/v3/users => generated 290 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 09:01:31.884309 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d646fc1-5a6b-46e8-a895-0ed9cf17d594 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:31.898926 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d646fc1-5a6b-46e8-a895-0ed9cf17d594 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:31.904237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b448e-23b9-4768-b29f-4d3c04004a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:31.904886 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b448e-23b9-4768-b29f-4d3c04004a32 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:31.905356 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b448e-23b9-4768-b29f-4d3c04004a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:31.905644 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b448e-23b9-4768-b29f-4d3c04004a32 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:31.906782 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b448e-23b9-4768-b29f-4d3c04004a32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:31.909188 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b448e-23b9-4768-b29f-4d3c04004a32 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:31.928865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1337/2660] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:31.932371 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d1c12-dc11-4db2-a516-2ec6f542bcb5 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:31.949548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d1c12-dc11-4db2-a516-2ec6f542bcb5 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:31.957200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dcf741-849f-4b47-98aa-1630c1f1dfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:31.957918 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dcf741-849f-4b47-98aa-1630c1f1dfea admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:31.958364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dcf741-849f-4b47-98aa-1630c1f1dfea admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:31.958779 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dcf741-849f-4b47-98aa-1630c1f1dfea admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:31.959582 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dcf741-849f-4b47-98aa-1630c1f1dfea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:31.961374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dcf741-849f-4b47-98aa-1630c1f1dfea admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:31.985036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1324/2661] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:31.985695 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b448e-23b9-4768-b29f-4d3c04004a32 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:32.001583 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b448e-23b9-4768-b29f-4d3c04004a32 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:32.011350 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeedb0e-79b7-46c9-9fa5-e55c5bff2592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:32.013072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeedb0e-79b7-46c9-9fa5-e55c5bff2592 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:32.013816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeedb0e-79b7-46c9-9fa5-e55c5bff2592 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:32.014787 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeedb0e-79b7-46c9-9fa5-e55c5bff2592 admin admin] PATH_INFO: `/v3/projects/bf65a59afdd949a9b4a36d389dc4748e/users/744980124dc041bbb94b5128796217a7/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:32.035993 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeedb0e-79b7-46c9-9fa5-e55c5bff2592 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf65a59afdd949a9b4a36d389dc4748e, user_id=744980124dc041bbb94b5128796217a7, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:32.040439 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeedb0e-79b7-46c9-9fa5-e55c5bff2592 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:32.055574 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dcf741-849f-4b47-98aa-1630c1f1dfea admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:32.058083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-50dcf741-849f-4b47-98aa-1630c1f1dfea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:32.066970 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1338/2662] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:01:31 2022] PUT /identity/v3/projects/bf65a59afdd949a9b4a36d389dc4748e/users/744980124dc041bbb94b5128796217a7/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 09:01:32.079921 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7b025-32d7-4d07-9fa1-0f9d903aa14c None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:32.080197 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7b025-32d7-4d07-9fa1-0f9d903aa14c None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:32.080763 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7b025-32d7-4d07-9fa1-0f9d903aa14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:32.137013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ede7b025-32d7-4d07-9fa1-0f9d903aa14c None None] MFA Rules not processed for user `744980124dc041bbb94b5128796217a7`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:32.138072 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ede7b025-32d7-4d07-9fa1-0f9d903aa14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:32.157038 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dcf741-849f-4b47-98aa-1630c1f1dfea admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:32.240119 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0928604-2bd4-4262-b504-33ae1c185e2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:32.243530 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0928604-2bd4-4262-b504-33ae1c185e2c service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:32.243771 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0928604-2bd4-4262-b504-33ae1c185e2c service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:32.244048 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0928604-2bd4-4262-b504-33ae1c185e2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:32.246786 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a0928604-2bd4-4262-b504-33ae1c185e2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:32.264093 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1339/2663] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 09:01:32.272753 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7b025-32d7-4d07-9fa1-0f9d903aa14c None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:32.273527 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:32.383550 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0928604-2bd4-4262-b504-33ae1c185e2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:32.385014 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0928604-2bd4-4262-b504-33ae1c185e2c service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:32.399266 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1325/2664] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:01:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 09:01:32.420513 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7b025-32d7-4d07-9fa1-0f9d903aa14c None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:32.427979 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaafe1f-f9ea-4b1f-94b6-8b54e1349ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:32.429835 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaafe1f-f9ea-4b1f-94b6-8b54e1349ad2 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:32.430268 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaafe1f-f9ea-4b1f-94b6-8b54e1349ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:32.430738 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaafe1f-f9ea-4b1f-94b6-8b54e1349ad2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:32.431921 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaafe1f-f9ea-4b1f-94b6-8b54e1349ad2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:32.435083 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaafe1f-f9ea-4b1f-94b6-8b54e1349ad2 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:32.477904 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1340/2665] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:01:32 2022] POST /identity/v3/projects => generated 377 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 09:01:32.479411 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0928604-2bd4-4262-b504-33ae1c185e2c service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:32.491737 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0928604-2bd4-4262-b504-33ae1c185e2c service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:32.576167 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437d98e-a71f-4428-abd8-f85661b001a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:32.576926 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437d98e-a71f-4428-abd8-f85661b001a7 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:32.577282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437d98e-a71f-4428-abd8-f85661b001a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:32.577624 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2437d98e-a71f-4428-abd8-f85661b001a7 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:32.578385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2437d98e-a71f-4428-abd8-f85661b001a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:32.580671 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2437d98e-a71f-4428-abd8-f85661b001a7 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:32.618993 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1326/2666] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:01:32 2022] POST /identity/v3/users => generated 289 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:01:32.623029 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaafe1f-f9ea-4b1f-94b6-8b54e1349ad2 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:32.632402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:32.640628 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaafe1f-f9ea-4b1f-94b6-8b54e1349ad2 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:32.648689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd99b9-0d55-47df-bb3f-b370a29ad5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:32.648945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd99b9-0d55-47df-bb3f-b370a29ad5d6 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:32.649148 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd99b9-0d55-47df-bb3f-b370a29ad5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:32.649427 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd99b9-0d55-47df-bb3f-b370a29ad5d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:32.650360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd99b9-0d55-47df-bb3f-b370a29ad5d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:32.654543 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd99b9-0d55-47df-bb3f-b370a29ad5d6 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:32.667491 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1341/2667] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:32.675071 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437d98e-a71f-4428-abd8-f85661b001a7 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:32.696554 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2437d98e-a71f-4428-abd8-f85661b001a7 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:32.702569 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cef0bb-053f-40fe-b5c0-33d5e0af9701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:32.703401 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cef0bb-053f-40fe-b5c0-33d5e0af9701 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:32.703682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cef0bb-053f-40fe-b5c0-33d5e0af9701 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:32.704012 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cef0bb-053f-40fe-b5c0-33d5e0af9701 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:32.704675 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cef0bb-053f-40fe-b5c0-33d5e0af9701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:32.706451 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cef0bb-053f-40fe-b5c0-33d5e0af9701 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:32.721631 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1327/2668] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:32.723492 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd99b9-0d55-47df-bb3f-b370a29ad5d6 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:32.735448 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd99b9-0d55-47df-bb3f-b370a29ad5d6 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:32.741987 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ebebda-8e03-41ab-b3c6-a62218265bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:32.742675 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebebda-8e03-41ab-b3c6-a62218265bfa admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:32.743019 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebebda-8e03-41ab-b3c6-a62218265bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:32.743296 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebebda-8e03-41ab-b3c6-a62218265bfa admin admin] PATH_INFO: `/v3/projects/567fc69c7d874ad090391f4cea06e759/users/b45d3e9a2e0c4c299b1fe089124391d2/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:32.764247 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ebebda-8e03-41ab-b3c6-a62218265bfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=567fc69c7d874ad090391f4cea06e759, user_id=b45d3e9a2e0c4c299b1fe089124391d2, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:32.767213 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ebebda-8e03-41ab-b3c6-a62218265bfa admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:32.786102 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1342/2669] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:01:32 2022] PUT /identity/v3/projects/567fc69c7d874ad090391f4cea06e759/users/b45d3e9a2e0c4c299b1fe089124391d2/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 09:01:32.796690 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa97fc-991f-4398-ae55-b9330bf15a1b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:32.796981 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa97fc-991f-4398-ae55-b9330bf15a1b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:32.797263 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa97fc-991f-4398-ae55-b9330bf15a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:32.834152 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-4cfa97fc-991f-4398-ae55-b9330bf15a1b None None] MFA Rules not processed for user `b45d3e9a2e0c4c299b1fe089124391d2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:32.835466 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4cfa97fc-991f-4398-ae55-b9330bf15a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:32.953031 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1328/2670] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:32.958336 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ebebda-8e03-41ab-b3c6-a62218265bfa admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:33.048240 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ebebda-8e03-41ab-b3c6-a62218265bfa admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:33.058145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace2270-e649-4714-8e44-7fc8d7a6bc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:33.058145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace2270-e649-4714-8e44-7fc8d7a6bc6e admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:33.058145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace2270-e649-4714-8e44-7fc8d7a6bc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:33.058145 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace2270-e649-4714-8e44-7fc8d7a6bc6e admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:33.058502 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace2270-e649-4714-8e44-7fc8d7a6bc6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:33.059931 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace2270-e649-4714-8e44-7fc8d7a6bc6e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:33.102939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1343/2671] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:01:32 2022] POST /identity/v3/projects => generated 377 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:01:33.107868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa97fc-991f-4398-ae55-b9330bf15a1b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:33.123949 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa97fc-991f-4398-ae55-b9330bf15a1b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:33.129295 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:33.213156 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feddf50-011d-44f1-b23a-c2f100d651b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:33.213871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feddf50-011d-44f1-b23a-c2f100d651b6 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:33.214150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feddf50-011d-44f1-b23a-c2f100d651b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:33.214482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feddf50-011d-44f1-b23a-c2f100d651b6 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:33.215149 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feddf50-011d-44f1-b23a-c2f100d651b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:33.217264 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feddf50-011d-44f1-b23a-c2f100d651b6 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:33.254975 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1329/2672] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:01:33 2022] POST /identity/v3/users => generated 295 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 09:01:33.258398 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace2270-e649-4714-8e44-7fc8d7a6bc6e admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:33.270223 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace2270-e649-4714-8e44-7fc8d7a6bc6e admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:33.275190 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50dc08-fa91-4327-bfb5-977ac8e1e23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:33.275824 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50dc08-fa91-4327-bfb5-977ac8e1e23e admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:33.276075 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50dc08-fa91-4327-bfb5-977ac8e1e23e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:33.276377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50dc08-fa91-4327-bfb5-977ac8e1e23e admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:33.277012 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e50dc08-fa91-4327-bfb5-977ac8e1e23e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:33.278656 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e50dc08-fa91-4327-bfb5-977ac8e1e23e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:33.293498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1344/2673] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:33.297809 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feddf50-011d-44f1-b23a-c2f100d651b6 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:33.309100 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feddf50-011d-44f1-b23a-c2f100d651b6 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:33.313943 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0d064-bf7e-4aab-ab0f-d2dd9bccaa05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:33.315040 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0d064-bf7e-4aab-ab0f-d2dd9bccaa05 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:33.315040 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0d064-bf7e-4aab-ab0f-d2dd9bccaa05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:33.315471 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0d064-bf7e-4aab-ab0f-d2dd9bccaa05 admin admin] PATH_INFO: `/v3/projects/488213d9bef1484993521325ad7dd2fc/users/b84b0803a6bd4caead91a33aff2af9b6/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:33.335303 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b0d064-bf7e-4aab-ab0f-d2dd9bccaa05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=488213d9bef1484993521325ad7dd2fc, user_id=b84b0803a6bd4caead91a33aff2af9b6, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:33.338855 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b0d064-bf7e-4aab-ab0f-d2dd9bccaa05 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:33.367712 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1330/2674] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:01:33 2022] PUT /identity/v3/projects/488213d9bef1484993521325ad7dd2fc/users/b84b0803a6bd4caead91a33aff2af9b6/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:33.371559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50dc08-fa91-4327-bfb5-977ac8e1e23e admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:33.384274 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50dc08-fa91-4327-bfb5-977ac8e1e23e admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:33.477558 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb92214a-779e-4d72-9f64-5e9bbed1eabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:33.478088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92214a-779e-4d72-9f64-5e9bbed1eabb admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:33.478390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92214a-779e-4d72-9f64-5e9bbed1eabb admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:33.478666 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92214a-779e-4d72-9f64-5e9bbed1eabb admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:33.479363 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb92214a-779e-4d72-9f64-5e9bbed1eabb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:33.481054 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb92214a-779e-4d72-9f64-5e9bbed1eabb admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:33.496063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1345/2675] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:33.503578 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0d064-bf7e-4aab-ab0f-d2dd9bccaa05 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:33.520133 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0d064-bf7e-4aab-ab0f-d2dd9bccaa05 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:33.527585 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d1026-8415-4794-b968-79f930eb60f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:33.528801 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d1026-8415-4794-b968-79f930eb60f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:33.529297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d1026-8415-4794-b968-79f930eb60f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:33.529829 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d1026-8415-4794-b968-79f930eb60f6 admin admin] PATH_INFO: `/v3/projects/488213d9bef1484993521325ad7dd2fc/users/b84b0803a6bd4caead91a33aff2af9b6/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:33.533027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d1026-8415-4794-b968-79f930eb60f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=488213d9bef1484993521325ad7dd2fc, user_id=b84b0803a6bd4caead91a33aff2af9b6, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:33.538732 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d1026-8415-4794-b968-79f930eb60f6 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:33.556358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1331/2676] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:01:33 2022] PUT /identity/v3/projects/488213d9bef1484993521325ad7dd2fc/users/b84b0803a6bd4caead91a33aff2af9b6/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 09:01:33.567736 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5828c0-cd47-4012-88ab-3fa8ce186481 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:33.568321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5828c0-cd47-4012-88ab-3fa8ce186481 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:33.568744 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5828c0-cd47-4012-88ab-3fa8ce186481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:33.604465 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-6e5828c0-cd47-4012-88ab-3fa8ce186481 None None] MFA Rules not processed for user `b84b0803a6bd4caead91a33aff2af9b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:33.605781 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-6e5828c0-cd47-4012-88ab-3fa8ce186481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:33.648473 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:33.722591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1346/2677] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:33.730621 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d1026-8415-4794-b968-79f930eb60f6 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:33.732385 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c91d1026-8415-4794-b968-79f930eb60f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:33.830696 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d1026-8415-4794-b968-79f930eb60f6 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:33.836866 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5828c0-cd47-4012-88ab-3fa8ce186481 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:33.838068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-6e5828c0-cd47-4012-88ab-3fa8ce186481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:33.840857 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf75c0-7869-4e62-aef1-d1ad2245f21d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:33.841512 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf75c0-7869-4e62-aef1-d1ad2245f21d service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:33.841840 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf75c0-7869-4e62-aef1-d1ad2245f21d service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:33.842659 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf75c0-7869-4e62-aef1-d1ad2245f21d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:33.853326 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cf75c0-7869-4e62-aef1-d1ad2245f21d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:33.855361 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cf75c0-7869-4e62-aef1-d1ad2245f21d service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:33.869662 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1332/2678] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:01:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:01:33.927557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf75c0-7869-4e62-aef1-d1ad2245f21d service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:33.938066 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf75c0-7869-4e62-aef1-d1ad2245f21d service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:33.939475 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5828c0-cd47-4012-88ab-3fa8ce186481 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:33.943287 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83daaf2b-071d-4192-bc38-4a937f9f042f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:33.943970 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59143ee-0cfb-48ba-bc36-02a0b584f6d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:33.944316 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83daaf2b-071d-4192-bc38-4a937f9f042f service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:33.944613 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59143ee-0cfb-48ba-bc36-02a0b584f6d0 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:33.944799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83daaf2b-071d-4192-bc38-4a937f9f042f service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:33.944991 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59143ee-0cfb-48ba-bc36-02a0b584f6d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:33.945169 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83daaf2b-071d-4192-bc38-4a937f9f042f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:33.945360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59143ee-0cfb-48ba-bc36-02a0b584f6d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:33.947393 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f59143ee-0cfb-48ba-bc36-02a0b584f6d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:34.024092 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83daaf2b-071d-4192-bc38-4a937f9f042f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:34.025601 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83daaf2b-071d-4192-bc38-4a937f9f042f service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:34.030836 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:34.036783 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1333/2679] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:01:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 09:01:34.045108 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59143ee-0cfb-48ba-bc36-02a0b584f6d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:34.046804 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59143ee-0cfb-48ba-bc36-02a0b584f6d0 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:34.060866 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1347/2680] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:01:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 09:01:35.797704 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83daaf2b-071d-4192-bc38-4a937f9f042f service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:35.810238 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83daaf2b-071d-4192-bc38-4a937f9f042f service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:35.814018 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c1652-4d8d-453d-ab9e-1d92fea00454 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:35.815113 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c1652-4d8d-453d-ab9e-1d92fea00454 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:35.816112 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c1652-4d8d-453d-ab9e-1d92fea00454 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:35.816892 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c1652-4d8d-453d-ab9e-1d92fea00454 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:35.819254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ad5c1652-4d8d-453d-ab9e-1d92fea00454 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:35.911054 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c1652-4d8d-453d-ab9e-1d92fea00454 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:35.914504 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c1652-4d8d-453d-ab9e-1d92fea00454 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:35.928654 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1334/2681] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:01:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:01:37.323284 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59143ee-0cfb-48ba-bc36-02a0b584f6d0 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:37.341715 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59143ee-0cfb-48ba-bc36-02a0b584f6d0 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:37.350544 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b02415-bd9c-42c3-975a-9cdbf20adea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:37.351299 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b02415-bd9c-42c3-975a-9cdbf20adea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:37.352783 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b02415-bd9c-42c3-975a-9cdbf20adea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:37.353253 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b02415-bd9c-42c3-975a-9cdbf20adea6 admin admin] PATH_INFO: `/v3/users/744980124dc041bbb94b5128796217a7` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:37.355051 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b02415-bd9c-42c3-975a-9cdbf20adea6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=744980124dc041bbb94b5128796217a7)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:37.357253 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b02415-bd9c-42c3-975a-9cdbf20adea6 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:37.557390 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f2b02415-bd9c-42c3-975a-9cdbf20adea6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '744980124dc041bbb94b5128796217a7'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:37.569561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:37.573001 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f2b02415-bd9c-42c3-975a-9cdbf20adea6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '744980124dc041bbb94b5128796217a7'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:37.591014 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f2b02415-bd9c-42c3-975a-9cdbf20adea6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '744980124dc041bbb94b5128796217a7'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:37.602859 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f2b02415-bd9c-42c3-975a-9cdbf20adea6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '744980124dc041bbb94b5128796217a7'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:37.603322 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f2b02415-bd9c-42c3-975a-9cdbf20adea6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '744980124dc041bbb94b5128796217a7'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:37.610035 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1348/2682] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:01:37 2022] DELETE /identity/v3/users/744980124dc041bbb94b5128796217a7 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:37.615731 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd8e0c-3ad4-4677-8763-c7cddffbe6f9 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:37.616604 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd8e0c-3ad4-4677-8763-c7cddffbe6f9 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:37.616957 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd8e0c-3ad4-4677-8763-c7cddffbe6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:37.658163 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-7fbd8e0c-3ad4-4677-8763-c7cddffbe6f9 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:37.659844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7fbd8e0c-3ad4-4677-8763-c7cddffbe6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:37.781143 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1335/2683] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:37.793766 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b02415-bd9c-42c3-975a-9cdbf20adea6 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:37.796235 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f2b02415-bd9c-42c3-975a-9cdbf20adea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:37.893694 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b02415-bd9c-42c3-975a-9cdbf20adea6 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:37.904489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5801f9-b8cd-43a5-a6be-b74e5438d23e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:37.908154 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5801f9-b8cd-43a5-a6be-b74e5438d23e service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:37.908494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5801f9-b8cd-43a5-a6be-b74e5438d23e service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:37.908842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5801f9-b8cd-43a5-a6be-b74e5438d23e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:37.921728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5801f9-b8cd-43a5-a6be-b74e5438d23e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:37.928293 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5801f9-b8cd-43a5-a6be-b74e5438d23e service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:37.941303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1349/2684] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:01:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:01:38.389623 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbd8e0c-3ad4-4677-8763-c7cddffbe6f9 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:38.391147 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-7fbd8e0c-3ad4-4677-8763-c7cddffbe6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:38.432620 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbd8e0c-3ad4-4677-8763-c7cddffbe6f9 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:38.446115 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1724b-19fe-4458-81db-9f1af85ca2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:38.446796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1724b-19fe-4458-81db-9f1af85ca2ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:38.447111 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1724b-19fe-4458-81db-9f1af85ca2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:38.447382 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1724b-19fe-4458-81db-9f1af85ca2ca admin admin] PATH_INFO: `/v3/projects/bf65a59afdd949a9b4a36d389dc4748e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:38.454507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1724b-19fe-4458-81db-9f1af85ca2ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf65a59afdd949a9b4a36d389dc4748e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:38.457561 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1724b-19fe-4458-81db-9f1af85ca2ca admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:38.566354 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-aef1724b-19fe-4458-81db-9f1af85ca2ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf65a59afdd949a9b4a36d389dc4748e', 'request_id': 'req-aef1724b-19fe-4458-81db-9f1af85ca2ca'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:38.579357 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-aef1724b-19fe-4458-81db-9f1af85ca2ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf65a59afdd949a9b4a36d389dc4748e', 'request_id': 'req-aef1724b-19fe-4458-81db-9f1af85ca2ca'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:38.590334 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-aef1724b-19fe-4458-81db-9f1af85ca2ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf65a59afdd949a9b4a36d389dc4748e', 'request_id': 'req-aef1724b-19fe-4458-81db-9f1af85ca2ca'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:38.611548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-aef1724b-19fe-4458-81db-9f1af85ca2ca admin admin] The token cache is being invalidate because project bf65a59afdd949a9b4a36d389dc4748e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:01:38.612071 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-aef1724b-19fe-4458-81db-9f1af85ca2ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:38.613655 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-aef1724b-19fe-4458-81db-9f1af85ca2ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:38.615443 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1336/2685] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:01:38 2022] DELETE /identity/v3/projects/bf65a59afdd949a9b4a36d389dc4748e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:38.620103 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5801f9-b8cd-43a5-a6be-b74e5438d23e service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:38.631195 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5801f9-b8cd-43a5-a6be-b74e5438d23e service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:38.709961 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8d29b-e1eb-486e-84dc-b92abd4b5b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:38.710756 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8d29b-e1eb-486e-84dc-b92abd4b5b82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:38.711255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8d29b-e1eb-486e-84dc-b92abd4b5b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:38.711794 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8d29b-e1eb-486e-84dc-b92abd4b5b82 admin admin] PATH_INFO: `/v3/users/b45d3e9a2e0c4c299b1fe089124391d2` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:38.713783 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8d29b-e1eb-486e-84dc-b92abd4b5b82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b45d3e9a2e0c4c299b1fe089124391d2)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:38.715963 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8d29b-e1eb-486e-84dc-b92abd4b5b82 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:38.815294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b9c8d29b-e1eb-486e-84dc-b92abd4b5b82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b45d3e9a2e0c4c299b1fe089124391d2'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:38.829074 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b9c8d29b-e1eb-486e-84dc-b92abd4b5b82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b45d3e9a2e0c4c299b1fe089124391d2'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:38.837796 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b9c8d29b-e1eb-486e-84dc-b92abd4b5b82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b45d3e9a2e0c4c299b1fe089124391d2'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:38.850100 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b9c8d29b-e1eb-486e-84dc-b92abd4b5b82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b45d3e9a2e0c4c299b1fe089124391d2'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:38.853273 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-b9c8d29b-e1eb-486e-84dc-b92abd4b5b82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b45d3e9a2e0c4c299b1fe089124391d2'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:38.858035 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:38.863417 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1350/2686] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:01:38 2022] DELETE /identity/v3/users/b45d3e9a2e0c4c299b1fe089124391d2 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:39.863641 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1724b-19fe-4458-81db-9f1af85ca2ca admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:39.865351 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-aef1724b-19fe-4458-81db-9f1af85ca2ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:39.887587 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1724b-19fe-4458-81db-9f1af85ca2ca admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:39.954155 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:40.040596 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3837995-bc71-4d23-8942-8065a1b9d85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:40.041484 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3837995-bc71-4d23-8942-8065a1b9d85b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:40.041855 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3837995-bc71-4d23-8942-8065a1b9d85b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:40.042313 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3837995-bc71-4d23-8942-8065a1b9d85b admin admin] PATH_INFO: `/v3/projects/567fc69c7d874ad090391f4cea06e759` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:40.044873 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3837995-bc71-4d23-8942-8065a1b9d85b admin admin] RBAC: Authorizing `identity:delete_project(project_id=567fc69c7d874ad090391f4cea06e759)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:40.049076 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3837995-bc71-4d23-8942-8065a1b9d85b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:40.138247 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b3837995-bc71-4d23-8942-8065a1b9d85b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '567fc69c7d874ad090391f4cea06e759', 'request_id': 'req-b3837995-bc71-4d23-8942-8065a1b9d85b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:40.159907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b3837995-bc71-4d23-8942-8065a1b9d85b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '567fc69c7d874ad090391f4cea06e759', 'request_id': 'req-b3837995-bc71-4d23-8942-8065a1b9d85b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:40.174005 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b3837995-bc71-4d23-8942-8065a1b9d85b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '567fc69c7d874ad090391f4cea06e759', 'request_id': 'req-b3837995-bc71-4d23-8942-8065a1b9d85b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:40.223203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b3837995-bc71-4d23-8942-8065a1b9d85b admin admin] The token cache is being invalidate because project 567fc69c7d874ad090391f4cea06e759 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:01:40.224710 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b3837995-bc71-4d23-8942-8065a1b9d85b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:40.226257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-b3837995-bc71-4d23-8942-8065a1b9d85b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:40.230477 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1337/2687] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:01:39 2022] DELETE /identity/v3/projects/567fc69c7d874ad090391f4cea06e759 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:40.234432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8d29b-e1eb-486e-84dc-b92abd4b5b82 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:40.236005 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b9c8d29b-e1eb-486e-84dc-b92abd4b5b82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:40.261864 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8d29b-e1eb-486e-84dc-b92abd4b5b82 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:40.388096 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f915d-730a-40f4-b22e-32fdfffd12b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:40.388775 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f915d-730a-40f4-b22e-32fdfffd12b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:40.389053 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f915d-730a-40f4-b22e-32fdfffd12b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:40.389530 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f915d-730a-40f4-b22e-32fdfffd12b5 admin admin] PATH_INFO: `/v3/users/b84b0803a6bd4caead91a33aff2af9b6` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:40.392269 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f915d-730a-40f4-b22e-32fdfffd12b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b84b0803a6bd4caead91a33aff2af9b6)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:40.394338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f915d-730a-40f4-b22e-32fdfffd12b5 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:40.488217 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-692f915d-730a-40f4-b22e-32fdfffd12b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b84b0803a6bd4caead91a33aff2af9b6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:40.518863 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-692f915d-730a-40f4-b22e-32fdfffd12b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b84b0803a6bd4caead91a33aff2af9b6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:40.531484 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-692f915d-730a-40f4-b22e-32fdfffd12b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b84b0803a6bd4caead91a33aff2af9b6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:40.548126 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-692f915d-730a-40f4-b22e-32fdfffd12b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b84b0803a6bd4caead91a33aff2af9b6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:40.548969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-692f915d-730a-40f4-b22e-32fdfffd12b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b84b0803a6bd4caead91a33aff2af9b6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:40.557037 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1351/2688] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:01:40 2022] DELETE /identity/v3/users/b84b0803a6bd4caead91a33aff2af9b6 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:40.675638 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3837995-bc71-4d23-8942-8065a1b9d85b admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:40.677552 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b3837995-bc71-4d23-8942-8065a1b9d85b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:40.706025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3837995-bc71-4d23-8942-8065a1b9d85b admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:40.882464 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957590a5-e7ef-42f6-b8a0-866443d3ec5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:40.884605 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957590a5-e7ef-42f6-b8a0-866443d3ec5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:40.885084 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957590a5-e7ef-42f6-b8a0-866443d3ec5c admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:40.886711 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957590a5-e7ef-42f6-b8a0-866443d3ec5c admin admin] PATH_INFO: `/v3/users/406be06d6adf40bbb48d9baf12022988` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:40.889083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957590a5-e7ef-42f6-b8a0-866443d3ec5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=406be06d6adf40bbb48d9baf12022988)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:40.900979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957590a5-e7ef-42f6-b8a0-866443d3ec5c admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:41.055419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f915d-730a-40f4-b22e-32fdfffd12b5 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:41.058678 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-692f915d-730a-40f4-b22e-32fdfffd12b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:41.092087 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f915d-730a-40f4-b22e-32fdfffd12b5 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:41.106213 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b489d-deeb-48ed-a864-8fd8b3f71994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:41.107094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b489d-deeb-48ed-a864-8fd8b3f71994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:41.107491 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b489d-deeb-48ed-a864-8fd8b3f71994 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:41.107867 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b489d-deeb-48ed-a864-8fd8b3f71994 admin admin] PATH_INFO: `/v3/projects/488213d9bef1484993521325ad7dd2fc` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:41.109673 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b489d-deeb-48ed-a864-8fd8b3f71994 admin admin] RBAC: Authorizing `identity:delete_project(project_id=488213d9bef1484993521325ad7dd2fc)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:41.112103 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b489d-deeb-48ed-a864-8fd8b3f71994 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:41.132662 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-957590a5-e7ef-42f6-b8a0-866443d3ec5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '406be06d6adf40bbb48d9baf12022988'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:41.135893 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:41.143642 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-957590a5-e7ef-42f6-b8a0-866443d3ec5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '406be06d6adf40bbb48d9baf12022988'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:41.154483 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-957590a5-e7ef-42f6-b8a0-866443d3ec5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '406be06d6adf40bbb48d9baf12022988'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:41.155395 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-957590a5-e7ef-42f6-b8a0-866443d3ec5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '406be06d6adf40bbb48d9baf12022988'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:41.156235 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-957590a5-e7ef-42f6-b8a0-866443d3ec5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '406be06d6adf40bbb48d9baf12022988'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:41.187093 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1338/2689] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:01:40 2022] DELETE /identity/v3/users/406be06d6adf40bbb48d9baf12022988 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:41.201488 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-1f1b489d-deeb-48ed-a864-8fd8b3f71994 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '488213d9bef1484993521325ad7dd2fc', 'request_id': 'req-1f1b489d-deeb-48ed-a864-8fd8b3f71994'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:41.207995 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-1f1b489d-deeb-48ed-a864-8fd8b3f71994 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '488213d9bef1484993521325ad7dd2fc', 'request_id': 'req-1f1b489d-deeb-48ed-a864-8fd8b3f71994'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:41.223959 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-1f1b489d-deeb-48ed-a864-8fd8b3f71994 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '488213d9bef1484993521325ad7dd2fc', 'request_id': 'req-1f1b489d-deeb-48ed-a864-8fd8b3f71994'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:41.240919 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-1f1b489d-deeb-48ed-a864-8fd8b3f71994 admin admin] The token cache is being invalidate because project 488213d9bef1484993521325ad7dd2fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:01:41.241847 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-1f1b489d-deeb-48ed-a864-8fd8b3f71994 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:41.243045 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-1f1b489d-deeb-48ed-a864-8fd8b3f71994 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:41.247169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1352/2690] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:01:41 2022] DELETE /identity/v3/projects/488213d9bef1484993521325ad7dd2fc => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:41.256949 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a7d60-b7bd-4d0c-8d00-0871c73f480e None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:41.257240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a7d60-b7bd-4d0c-8d00-0871c73f480e None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:41.257513 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a7d60-b7bd-4d0c-8d00-0871c73f480e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:41.278986 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b93a7d60-b7bd-4d0c-8d00-0871c73f480e None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:41.280133 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b93a7d60-b7bd-4d0c-8d00-0871c73f480e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:41.349180 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1339/2691] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:41.360588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47ca20-3499-4c9d-9bdc-f2074b2b4e57 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:41.360989 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47ca20-3499-4c9d-9bdc-f2074b2b4e57 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:41.361873 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47ca20-3499-4c9d-9bdc-f2074b2b4e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:41.378951 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:41.403451 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-5a47ca20-3499-4c9d-9bdc-f2074b2b4e57 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:41.404720 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-5a47ca20-3499-4c9d-9bdc-f2074b2b4e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:41.424810 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1353/2692] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:41.426487 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a7d60-b7bd-4d0c-8d00-0871c73f480e None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:41.428365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b93a7d60-b7bd-4d0c-8d00-0871c73f480e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:41.444334 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47ca20-3499-4c9d-9bdc-f2074b2b4e57 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:41.457335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a7d60-b7bd-4d0c-8d00-0871c73f480e None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:41.459295 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47ca20-3499-4c9d-9bdc-f2074b2b4e57 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:41.474240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4bf8a0-2504-49ae-bb55-388a63868390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:41.475167 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4bf8a0-2504-49ae-bb55-388a63868390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:41.475592 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4bf8a0-2504-49ae-bb55-388a63868390 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:41.476031 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4bf8a0-2504-49ae-bb55-388a63868390 admin admin] PATH_INFO: `/v3/projects/f87dce6d7f01455fb8c3cfe128c20e72` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:41.478155 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4bf8a0-2504-49ae-bb55-388a63868390 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f87dce6d7f01455fb8c3cfe128c20e72)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:41.487194 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4470ccbb-d2de-49b2-ba13-5140f7cfb1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:41.488123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4470ccbb-d2de-49b2-ba13-5140f7cfb1e2 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:41.490126 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4470ccbb-d2de-49b2-ba13-5140f7cfb1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:41.490247 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4bf8a0-2504-49ae-bb55-388a63868390 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:41.490709 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4470ccbb-d2de-49b2-ba13-5140f7cfb1e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:41.491789 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4470ccbb-d2de-49b2-ba13-5140f7cfb1e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 09:01:41.492730 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4470ccbb-d2de-49b2-ba13-5140f7cfb1e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:41.497291 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4470ccbb-d2de-49b2-ba13-5140f7cfb1e2 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:41.528892 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1354/2693] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 09:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 09:01:41.535077 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4470ccbb-d2de-49b2-ba13-5140f7cfb1e2 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:41.549832 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4470ccbb-d2de-49b2-ba13-5140f7cfb1e2 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:41.568981 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3a4bf8a0-2504-49ae-bb55-388a63868390 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f87dce6d7f01455fb8c3cfe128c20e72', 'request_id': 'req-3a4bf8a0-2504-49ae-bb55-388a63868390'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:41.578603 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3a4bf8a0-2504-49ae-bb55-388a63868390 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f87dce6d7f01455fb8c3cfe128c20e72', 'request_id': 'req-3a4bf8a0-2504-49ae-bb55-388a63868390'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:41.595627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3a4bf8a0-2504-49ae-bb55-388a63868390 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f87dce6d7f01455fb8c3cfe128c20e72', 'request_id': 'req-3a4bf8a0-2504-49ae-bb55-388a63868390'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:41.611398 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3a4bf8a0-2504-49ae-bb55-388a63868390 admin admin] The token cache is being invalidate because project f87dce6d7f01455fb8c3cfe128c20e72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:01:41.611804 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3a4bf8a0-2504-49ae-bb55-388a63868390 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:41.613476 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3a4bf8a0-2504-49ae-bb55-388a63868390 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:41.614173 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1340/2694] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:01:41 2022] DELETE /identity/v3/projects/f87dce6d7f01455fb8c3cfe128c20e72 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:41.620734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2de70-169d-4976-b0a2-b344464e2a1f None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:41.621025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2de70-169d-4976-b0a2-b344464e2a1f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:41.621299 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2de70-169d-4976-b0a2-b344464e2a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:41.644896 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b0c2de70-169d-4976-b0a2-b344464e2a1f None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:41.646407 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b0c2de70-169d-4976-b0a2-b344464e2a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:41.660787 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d3418-d52d-4821-960e-60a39defe672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:41.661510 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d3418-d52d-4821-960e-60a39defe672 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:41.662342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d3418-d52d-4821-960e-60a39defe672 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:41.662342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d3418-d52d-4821-960e-60a39defe672 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:41.663038 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d3418-d52d-4821-960e-60a39defe672 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:41.665156 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d3418-d52d-4821-960e-60a39defe672 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:41.700157 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1355/2695] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:01:41 2022] POST /identity/v3/projects => generated 375 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:01:41.704360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d3418-d52d-4821-960e-60a39defe672 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:41.707286 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ee9d3418-d52d-4821-960e-60a39defe672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:41.714351 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:41.745762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1341/2696] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:41.755029 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0376d-8955-4411-a733-864193385642 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:41.755398 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0376d-8955-4411-a733-864193385642 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:41.756181 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0376d-8955-4411-a733-864193385642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:41.797887 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-aff0376d-8955-4411-a733-864193385642 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:41.800000 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-aff0376d-8955-4411-a733-864193385642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:41.832106 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d3418-d52d-4821-960e-60a39defe672 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:41.837397 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e599054e-c569-4f30-a9df-51e50e7a62a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:41.838310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599054e-c569-4f30-a9df-51e50e7a62a6 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:41.838310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599054e-c569-4f30-a9df-51e50e7a62a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:41.838763 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599054e-c569-4f30-a9df-51e50e7a62a6 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:41.839249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e599054e-c569-4f30-a9df-51e50e7a62a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:41.841265 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e599054e-c569-4f30-a9df-51e50e7a62a6 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:41.866215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1356/2697] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:01:41 2022] POST /identity/v3/users => generated 288 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 09:01:41.870314 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e599054e-c569-4f30-a9df-51e50e7a62a6 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:41.871101 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1342/2698] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:41.875267 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0376d-8955-4411-a733-864193385642 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:41.886151 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0376d-8955-4411-a733-864193385642 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:41.887282 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e599054e-c569-4f30-a9df-51e50e7a62a6 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:41.892232 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ce745-cb54-4abf-8b1b-a124b9688bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:41.892964 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ce745-cb54-4abf-8b1b-a124b9688bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:41.893143 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339211a-cea6-4574-b4a6-a8a179697444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:41.893549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ce745-cb54-4abf-8b1b-a124b9688bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:41.893820 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339211a-cea6-4574-b4a6-a8a179697444 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:41.894154 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339211a-cea6-4574-b4a6-a8a179697444 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:41.894334 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ce745-cb54-4abf-8b1b-a124b9688bc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:41.894494 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339211a-cea6-4574-b4a6-a8a179697444 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:41.895105 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0339211a-cea6-4574-b4a6-a8a179697444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:41.895314 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ce745-cb54-4abf-8b1b-a124b9688bc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 09:01:41.895636 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ce745-cb54-4abf-8b1b-a124b9688bc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:41.897067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0339211a-cea6-4574-b4a6-a8a179697444 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:41.897425 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ce745-cb54-4abf-8b1b-a124b9688bc1 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:41.910461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1357/2699] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:41.915791 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339211a-cea6-4574-b4a6-a8a179697444 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:41.916874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1343/2700] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 09:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 09:01:41.923124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ce745-cb54-4abf-8b1b-a124b9688bc1 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:41.936795 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ce745-cb54-4abf-8b1b-a124b9688bc1 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:41.942756 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5bb873-89fe-4b08-a7c8-ddf694e07ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:41.943531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bb873-89fe-4b08-a7c8-ddf694e07ad3 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:41.943969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bb873-89fe-4b08-a7c8-ddf694e07ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:41.944252 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bb873-89fe-4b08-a7c8-ddf694e07ad3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:41.944902 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5bb873-89fe-4b08-a7c8-ddf694e07ad3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:41.946539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339211a-cea6-4574-b4a6-a8a179697444 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:41.947350 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5bb873-89fe-4b08-a7c8-ddf694e07ad3 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:41.957172 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b35079-9f24-45c0-a42c-f352523f8187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:41.958045 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b35079-9f24-45c0-a42c-f352523f8187 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:41.958483 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b35079-9f24-45c0-a42c-f352523f8187 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:41.958851 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b35079-9f24-45c0-a42c-f352523f8187 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:41.959728 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b35079-9f24-45c0-a42c-f352523f8187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:41.962865 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b35079-9f24-45c0-a42c-f352523f8187 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:41.978313 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1358/2701] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 09:01:41.985333 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b35079-9f24-45c0-a42c-f352523f8187 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:41.990562 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1344/2702] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:01:41 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:01:41.992063 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5bb873-89fe-4b08-a7c8-ddf694e07ad3 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:41.999238 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b35079-9f24-45c0-a42c-f352523f8187 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:42.053455 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:42.090174 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5bb873-89fe-4b08-a7c8-ddf694e07ad3 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:42.096569 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474c6e4-9431-40db-97f9-ee1fe0472aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:42.097219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474c6e4-9431-40db-97f9-ee1fe0472aba admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:42.097509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474c6e4-9431-40db-97f9-ee1fe0472aba admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:42.097784 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474c6e4-9431-40db-97f9-ee1fe0472aba admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:42.098433 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8474c6e4-9431-40db-97f9-ee1fe0472aba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:42.098771 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2310b-5708-450c-a8e9-2be384650995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:42.099816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2310b-5708-450c-a8e9-2be384650995 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:42.100103 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2310b-5708-450c-a8e9-2be384650995 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:42.100372 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2310b-5708-450c-a8e9-2be384650995 admin admin] PATH_INFO: `/v3/projects/683f469c5bf74200b35f846c4ad64d14/users/14b7a7358617411ea98023f054139bbc/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:42.100503 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8474c6e4-9431-40db-97f9-ee1fe0472aba admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:42.121089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2310b-5708-450c-a8e9-2be384650995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=683f469c5bf74200b35f846c4ad64d14, user_id=14b7a7358617411ea98023f054139bbc, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:42.123897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2310b-5708-450c-a8e9-2be384650995 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:42.145530 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1359/2703] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:01:41 2022] PUT /identity/v3/projects/683f469c5bf74200b35f846c4ad64d14/users/14b7a7358617411ea98023f054139bbc/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 09:01:42.146877 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1345/2704] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:01:41 2022] POST /identity/v3/users => generated 287 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:01:42.151290 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2310b-5708-450c-a8e9-2be384650995 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:42.158004 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace435f6-11b4-4b7c-961e-9c21ece9fa1d None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:42.158293 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace435f6-11b4-4b7c-961e-9c21ece9fa1d None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:42.158565 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace435f6-11b4-4b7c-961e-9c21ece9fa1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:42.198442 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-ace435f6-11b4-4b7c-961e-9c21ece9fa1d None None] MFA Rules not processed for user `14b7a7358617411ea98023f054139bbc`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:42.199901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ace435f6-11b4-4b7c-961e-9c21ece9fa1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:42.240250 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2310b-5708-450c-a8e9-2be384650995 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:42.246434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b1264-7c38-4d5e-bc73-4ab5ace8b497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:42.247357 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b1264-7c38-4d5e-bc73-4ab5ace8b497 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:42.247357 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b1264-7c38-4d5e-bc73-4ab5ace8b497 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:42.247628 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b1264-7c38-4d5e-bc73-4ab5ace8b497 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:42.248312 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305b1264-7c38-4d5e-bc73-4ab5ace8b497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:42.249070 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:42.249956 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305b1264-7c38-4d5e-bc73-4ab5ace8b497 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:42.263707 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1360/2705] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:42.268316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b1264-7c38-4d5e-bc73-4ab5ace8b497 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:42.279253 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b1264-7c38-4d5e-bc73-4ab5ace8b497 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:42.284425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32dd8a-d21e-4786-a577-3ce9c4a85b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:42.285088 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32dd8a-d21e-4786-a577-3ce9c4a85b02 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:42.285374 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32dd8a-d21e-4786-a577-3ce9c4a85b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:42.285671 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32dd8a-d21e-4786-a577-3ce9c4a85b02 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:42.286313 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c32dd8a-d21e-4786-a577-3ce9c4a85b02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:42.288028 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c32dd8a-d21e-4786-a577-3ce9c4a85b02 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:42.292160 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1346/2706] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:42.302395 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1361/2707] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 09:01:42.304860 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace435f6-11b4-4b7c-961e-9c21ece9fa1d None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:42.306664 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32dd8a-d21e-4786-a577-3ce9c4a85b02 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:42.306837 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-ace435f6-11b4-4b7c-961e-9c21ece9fa1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:42.318174 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32dd8a-d21e-4786-a577-3ce9c4a85b02 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:42.324330 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9697b37c-a345-49af-ad10-4f73fce63043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:42.325016 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9697b37c-a345-49af-ad10-4f73fce63043 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:42.325297 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9697b37c-a345-49af-ad10-4f73fce63043 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:42.325574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9697b37c-a345-49af-ad10-4f73fce63043 admin admin] PATH_INFO: `/v3/projects/7ca97fa879a446cbbed3fd5133ac0ac2/users/10af5c7026dd4aaaafb93bc6ce6005ad/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:42.348517 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9697b37c-a345-49af-ad10-4f73fce63043 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca97fa879a446cbbed3fd5133ac0ac2, user_id=10af5c7026dd4aaaafb93bc6ce6005ad, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:42.351450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9697b37c-a345-49af-ad10-4f73fce63043 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:42.372160 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1362/2708] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:01:42 2022] PUT /identity/v3/projects/7ca97fa879a446cbbed3fd5133ac0ac2/users/10af5c7026dd4aaaafb93bc6ce6005ad/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 09:01:42.384357 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947fadf-7b51-4579-a521-7ad15e0e4c18 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:42.384742 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947fadf-7b51-4579-a521-7ad15e0e4c18 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:42.384983 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947fadf-7b51-4579-a521-7ad15e0e4c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:42.400103 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace435f6-11b4-4b7c-961e-9c21ece9fa1d None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:42.423403 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-b947fadf-7b51-4579-a521-7ad15e0e4c18 None None] MFA Rules not processed for user `10af5c7026dd4aaaafb93bc6ce6005ad`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:42.424710 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b947fadf-7b51-4579-a521-7ad15e0e4c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:42.459279 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988bc12-c5af-4b18-a38a-dccddcc92b81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:42.460280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988bc12-c5af-4b18-a38a-dccddcc92b81 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:42.460661 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988bc12-c5af-4b18-a38a-dccddcc92b81 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:42.460939 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988bc12-c5af-4b18-a38a-dccddcc92b81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:42.503255 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1363/2709] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 09:01:42.516386 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947fadf-7b51-4579-a521-7ad15e0e4c18 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:42.517861 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b947fadf-7b51-4579-a521-7ad15e0e4c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:42.530294 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:42.537890 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988bc12-c5af-4b18-a38a-dccddcc92b81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:42.540047 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988bc12-c5af-4b18-a38a-dccddcc92b81 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:42.556116 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1347/2710] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:01:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:01:42.617034 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947fadf-7b51-4579-a521-7ad15e0e4c18 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:42.623607 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df495ab1-a086-4a44-bfca-48b9e76ae501 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:42.624231 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df495ab1-a086-4a44-bfca-48b9e76ae501 service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:42.624519 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df495ab1-a086-4a44-bfca-48b9e76ae501 service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:42.624783 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df495ab1-a086-4a44-bfca-48b9e76ae501 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:42.637003 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df495ab1-a086-4a44-bfca-48b9e76ae501 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:42.640602 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df495ab1-a086-4a44-bfca-48b9e76ae501 service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:42.655573 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1364/2711] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:01:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 09:01:42.935489 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988bc12-c5af-4b18-a38a-dccddcc92b81 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:43.011425 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=100 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:43.029379 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988bc12-c5af-4b18-a38a-dccddcc92b81 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:43.035081 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f011d9-48f7-4bf7-8b14-cdadbe4d8cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:43.035736 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f011d9-48f7-4bf7-8b14-cdadbe4d8cec admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:43.036015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f011d9-48f7-4bf7-8b14-cdadbe4d8cec admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:43.036285 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f011d9-48f7-4bf7-8b14-cdadbe4d8cec admin admin] PATH_INFO: `/v3/users/10af5c7026dd4aaaafb93bc6ce6005ad` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:43.037524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f011d9-48f7-4bf7-8b14-cdadbe4d8cec admin admin] RBAC: Authorizing `identity:delete_user(user_id=10af5c7026dd4aaaafb93bc6ce6005ad)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:43.039521 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f011d9-48f7-4bf7-8b14-cdadbe4d8cec admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:43.094456 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-30f011d9-48f7-4bf7-8b14-cdadbe4d8cec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10af5c7026dd4aaaafb93bc6ce6005ad'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:43.104225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-30f011d9-48f7-4bf7-8b14-cdadbe4d8cec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10af5c7026dd4aaaafb93bc6ce6005ad'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:43.113406 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-30f011d9-48f7-4bf7-8b14-cdadbe4d8cec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10af5c7026dd4aaaafb93bc6ce6005ad'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:43.115619 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-30f011d9-48f7-4bf7-8b14-cdadbe4d8cec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10af5c7026dd4aaaafb93bc6ce6005ad'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:43.117117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-30f011d9-48f7-4bf7-8b14-cdadbe4d8cec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10af5c7026dd4aaaafb93bc6ce6005ad'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:43.144140 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1348/2712] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:01:42 2022] DELETE /identity/v3/users/10af5c7026dd4aaaafb93bc6ce6005ad => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:43.149053 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d88f2-d59c-4c7c-acfd-2d8e29dbdec6 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:43.149342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d88f2-d59c-4c7c-acfd-2d8e29dbdec6 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:43.149690 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d88f2-d59c-4c7c-acfd-2d8e29dbdec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:43.174182 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-564d88f2-d59c-4c7c-acfd-2d8e29dbdec6 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:43.175496 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-564d88f2-d59c-4c7c-acfd-2d8e29dbdec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:43.253449 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1365/2713] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:43.268285 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f011d9-48f7-4bf7-8b14-cdadbe4d8cec admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:43.269859 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-30f011d9-48f7-4bf7-8b14-cdadbe4d8cec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:43.355596 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f011d9-48f7-4bf7-8b14-cdadbe4d8cec admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:43.360294 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d9f21-ce16-4439-82b4-4295d112b5ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:43.360924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d9f21-ce16-4439-82b4-4295d112b5ed service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:43.361196 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d9f21-ce16-4439-82b4-4295d112b5ed service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:43.361462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d9f21-ce16-4439-82b4-4295d112b5ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:43.375873 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d9f21-ce16-4439-82b4-4295d112b5ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:43.378256 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d9f21-ce16-4439-82b4-4295d112b5ed service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:43.395097 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1349/2714] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:01:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 09:01:43.714707 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d88f2-d59c-4c7c-acfd-2d8e29dbdec6 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:43.716447 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-564d88f2-d59c-4c7c-acfd-2d8e29dbdec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:43.732834 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d88f2-d59c-4c7c-acfd-2d8e29dbdec6 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:43.738784 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586491ba-6da3-47c7-8584-e33c39d995c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:43.739621 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586491ba-6da3-47c7-8584-e33c39d995c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:43.739763 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586491ba-6da3-47c7-8584-e33c39d995c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:43.740032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586491ba-6da3-47c7-8584-e33c39d995c0 admin admin] PATH_INFO: `/v3/projects/7ca97fa879a446cbbed3fd5133ac0ac2` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:43.741513 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586491ba-6da3-47c7-8584-e33c39d995c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ca97fa879a446cbbed3fd5133ac0ac2)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:43.743484 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586491ba-6da3-47c7-8584-e33c39d995c0 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:43.808063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-586491ba-6da3-47c7-8584-e33c39d995c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ca97fa879a446cbbed3fd5133ac0ac2', 'request_id': 'req-586491ba-6da3-47c7-8584-e33c39d995c0'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:43.815706 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-586491ba-6da3-47c7-8584-e33c39d995c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ca97fa879a446cbbed3fd5133ac0ac2', 'request_id': 'req-586491ba-6da3-47c7-8584-e33c39d995c0'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:43.838013 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-586491ba-6da3-47c7-8584-e33c39d995c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ca97fa879a446cbbed3fd5133ac0ac2', 'request_id': 'req-586491ba-6da3-47c7-8584-e33c39d995c0'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:43.855319 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-586491ba-6da3-47c7-8584-e33c39d995c0 admin admin] The token cache is being invalidate because project 7ca97fa879a446cbbed3fd5133ac0ac2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:01:43.856006 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-586491ba-6da3-47c7-8584-e33c39d995c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:43.857348 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-586491ba-6da3-47c7-8584-e33c39d995c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:43.860114 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1366/2715] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:01:43 2022] DELETE /identity/v3/projects/7ca97fa879a446cbbed3fd5133ac0ac2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:43.867863 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947f2ee-941e-4239-b835-b008f5f7d491 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:43.868279 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947f2ee-941e-4239-b835-b008f5f7d491 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:43.869105 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947f2ee-941e-4239-b835-b008f5f7d491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:43.902762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-6947f2ee-941e-4239-b835-b008f5f7d491 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:43.905038 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6947f2ee-941e-4239-b835-b008f5f7d491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:43.990840 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1350/2716] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:44.001749 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814e191-eefa-4dce-9df1-ee2bd27ec7f2 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:44.002053 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814e191-eefa-4dce-9df1-ee2bd27ec7f2 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:44.003293 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814e191-eefa-4dce-9df1-ee2bd27ec7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:44.031001 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-b814e191-eefa-4dce-9df1-ee2bd27ec7f2 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:44.032243 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b814e191-eefa-4dce-9df1-ee2bd27ec7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:44.042141 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1367/2717] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:44.047784 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947f2ee-941e-4239-b835-b008f5f7d491 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:44.059735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947f2ee-941e-4239-b835-b008f5f7d491 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:44.065279 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195455d0-ef8b-479a-9bb0-32c86455fd09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:44.066129 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195455d0-ef8b-479a-9bb0-32c86455fd09 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:44.066468 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195455d0-ef8b-479a-9bb0-32c86455fd09 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:44.066767 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195455d0-ef8b-479a-9bb0-32c86455fd09 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:44.067623 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195455d0-ef8b-479a-9bb0-32c86455fd09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 09:01:44.068061 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195455d0-ef8b-479a-9bb0-32c86455fd09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:44.069727 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195455d0-ef8b-479a-9bb0-32c86455fd09 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:44.073342 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:44.094231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1351/2718] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 09:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 09:01:44.096205 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b814e191-eefa-4dce-9df1-ee2bd27ec7f2 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:44.109156 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b814e191-eefa-4dce-9df1-ee2bd27ec7f2 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:44.115177 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f374a-c308-4eba-85ad-f7553a638902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:44.116036 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f374a-c308-4eba-85ad-f7553a638902 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:44.116542 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f374a-c308-4eba-85ad-f7553a638902 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:44.116780 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f374a-c308-4eba-85ad-f7553a638902 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:44.118271 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7f374a-c308-4eba-85ad-f7553a638902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:44.119817 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7f374a-c308-4eba-85ad-f7553a638902 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:44.133882 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:44.162814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1368/2719] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:01:44 2022] POST /identity/v3/projects => generated 405 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:01:44.167654 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195455d0-ef8b-479a-9bb0-32c86455fd09 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:44.180860 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195455d0-ef8b-479a-9bb0-32c86455fd09 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:44.266778 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e7a60-73ab-40cf-9576-10283b98969c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:44.267535 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e7a60-73ab-40cf-9576-10283b98969c admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:44.267839 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e7a60-73ab-40cf-9576-10283b98969c admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:44.268078 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e7a60-73ab-40cf-9576-10283b98969c admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:44.268829 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e7a60-73ab-40cf-9576-10283b98969c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:44.271122 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e7a60-73ab-40cf-9576-10283b98969c admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:44.316415 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1352/2720] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:01:44 2022] POST /identity/v3/users => generated 303 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 09:01:44.322104 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f374a-c308-4eba-85ad-f7553a638902 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:44.338984 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f374a-c308-4eba-85ad-f7553a638902 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:44.347202 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa82a8-75b0-430b-9118-b2f41e70dc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:44.347931 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa82a8-75b0-430b-9118-b2f41e70dc12 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:44.348376 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa82a8-75b0-430b-9118-b2f41e70dc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:44.348784 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa82a8-75b0-430b-9118-b2f41e70dc12 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:44.349566 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa82a8-75b0-430b-9118-b2f41e70dc12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:44.352336 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa82a8-75b0-430b-9118-b2f41e70dc12 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:44.367245 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1369/2721] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:44.372524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e7a60-73ab-40cf-9576-10283b98969c admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:44.386777 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e7a60-73ab-40cf-9576-10283b98969c admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:44.392184 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f9276-6915-4797-9fd7-1abcc105af38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:44.392823 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f9276-6915-4797-9fd7-1abcc105af38 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:44.393103 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f9276-6915-4797-9fd7-1abcc105af38 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:44.393462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f9276-6915-4797-9fd7-1abcc105af38 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:44.394178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f9276-6915-4797-9fd7-1abcc105af38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:44.395884 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f9276-6915-4797-9fd7-1abcc105af38 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:44.410767 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1353/2722] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:44.415976 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa82a8-75b0-430b-9118-b2f41e70dc12 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:44.428641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa82a8-75b0-430b-9118-b2f41e70dc12 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:44.433660 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876aed64-dde6-4855-9376-80c0f3d9bfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:44.434362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876aed64-dde6-4855-9376-80c0f3d9bfe9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:44.434636 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876aed64-dde6-4855-9376-80c0f3d9bfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:44.434943 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876aed64-dde6-4855-9376-80c0f3d9bfe9 admin admin] PATH_INFO: `/v3/projects/e5e1b236c287477fad700d5343b2a8a2/users/23c56ba26a0946cd9aff74c8bdf66582/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:44.456133 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876aed64-dde6-4855-9376-80c0f3d9bfe9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5e1b236c287477fad700d5343b2a8a2, user_id=23c56ba26a0946cd9aff74c8bdf66582, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:44.461598 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876aed64-dde6-4855-9376-80c0f3d9bfe9 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:44.485351 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1370/2723] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:01:44 2022] PUT /identity/v3/projects/e5e1b236c287477fad700d5343b2a8a2/users/23c56ba26a0946cd9aff74c8bdf66582/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:44.493051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580781b-5766-4d2f-9f8c-ec1eff54214b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:44.493421 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580781b-5766-4d2f-9f8c-ec1eff54214b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:44.493834 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580781b-5766-4d2f-9f8c-ec1eff54214b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:44.518247 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-3580781b-5766-4d2f-9f8c-ec1eff54214b None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:44.519635 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3580781b-5766-4d2f-9f8c-ec1eff54214b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:44.607698 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1354/2724] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:44.621482 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876aed64-dde6-4855-9376-80c0f3d9bfe9 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:44.623636 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-876aed64-dde6-4855-9376-80c0f3d9bfe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:44.708963 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876aed64-dde6-4855-9376-80c0f3d9bfe9 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:44.712249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c110736-c5a7-4949-8e37-44d929b2dbc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:44.712894 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c110736-c5a7-4949-8e37-44d929b2dbc0 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:44.713249 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c110736-c5a7-4949-8e37-44d929b2dbc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:44.713500 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c110736-c5a7-4949-8e37-44d929b2dbc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:44.724518 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c110736-c5a7-4949-8e37-44d929b2dbc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:44.725996 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c110736-c5a7-4949-8e37-44d929b2dbc0 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:44.742081 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1371/2725] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:01:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:01:44.985904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580781b-5766-4d2f-9f8c-ec1eff54214b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:45.000609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580781b-5766-4d2f-9f8c-ec1eff54214b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:45.007870 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe7dec-06c6-48fe-a2a9-8635866a550b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:45.008808 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe7dec-06c6-48fe-a2a9-8635866a550b service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:45.009934 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe7dec-06c6-48fe-a2a9-8635866a550b service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:45.010119 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe7dec-06c6-48fe-a2a9-8635866a550b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:45.013628 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-32fe7dec-06c6-48fe-a2a9-8635866a550b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:45.111310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fe7dec-06c6-48fe-a2a9-8635866a550b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:45.114046 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fe7dec-06c6-48fe-a2a9-8635866a550b service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:45.133727 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1355/2726] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:01:44 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:01:45.707947 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22369309-bb33-450b-8284-a30bf6219ace None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:45.708700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22369309-bb33-450b-8284-a30bf6219ace None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:45.708700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22369309-bb33-450b-8284-a30bf6219ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:45.747144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-22369309-bb33-450b-8284-a30bf6219ace None None] MFA Rules not processed for user `23c56ba26a0946cd9aff74c8bdf66582`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:45.749139 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-22369309-bb33-450b-8284-a30bf6219ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:45.862068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1372/2727] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:45 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:45.872679 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe7dec-06c6-48fe-a2a9-8635866a550b service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:45.888436 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe7dec-06c6-48fe-a2a9-8635866a550b service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:45.896303 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d244376-d85a-4f92-924a-fa3dac42db80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:45.896303 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d244376-d85a-4f92-924a-fa3dac42db80 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:45.896303 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d244376-d85a-4f92-924a-fa3dac42db80 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:45.896303 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d244376-d85a-4f92-924a-fa3dac42db80 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:45.896694 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d244376-d85a-4f92-924a-fa3dac42db80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:45.898097 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d244376-d85a-4f92-924a-fa3dac42db80 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:45.905276 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:45.943259 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1356/2728] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:01:45 2022] POST /identity/v3/projects => generated 403 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:01:45.944872 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22369309-bb33-450b-8284-a30bf6219ace None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:45.966373 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22369309-bb33-450b-8284-a30bf6219ace None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:46.048139 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:46.074623 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97607121-fd1f-442e-b9c1-97fa9b957889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:46.075430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97607121-fd1f-442e-b9c1-97fa9b957889 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:46.075715 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97607121-fd1f-442e-b9c1-97fa9b957889 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:46.075996 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97607121-fd1f-442e-b9c1-97fa9b957889 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:46.076657 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97607121-fd1f-442e-b9c1-97fa9b957889 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:46.078684 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97607121-fd1f-442e-b9c1-97fa9b957889 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:46.117003 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1373/2729] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:01:45 2022] POST /identity/v3/users => generated 308 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:01:46.120626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d244376-d85a-4f92-924a-fa3dac42db80 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:46.132706 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d244376-d85a-4f92-924a-fa3dac42db80 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:46.139042 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ffd92b-9565-4acb-b0da-cf772e37b8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:46.139682 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ffd92b-9565-4acb-b0da-cf772e37b8f1 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:46.139960 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ffd92b-9565-4acb-b0da-cf772e37b8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:46.140236 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ffd92b-9565-4acb-b0da-cf772e37b8f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:46.140960 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ffd92b-9565-4acb-b0da-cf772e37b8f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:46.142620 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ffd92b-9565-4acb-b0da-cf772e37b8f1 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:46.155388 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1357/2730] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:46.160545 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97607121-fd1f-442e-b9c1-97fa9b957889 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:46.173130 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97607121-fd1f-442e-b9c1-97fa9b957889 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:46.178441 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec69fa-1356-43b5-a30c-a73c111c98d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:46.179196 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec69fa-1356-43b5-a30c-a73c111c98d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:46.179547 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec69fa-1356-43b5-a30c-a73c111c98d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:46.179839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec69fa-1356-43b5-a30c-a73c111c98d7 admin admin] PATH_INFO: `/v3/projects/f6ae9a28746242afa489fc8650c2c632/users/618618df80e846449ab61d7daa875bfb/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:46.200510 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec69fa-1356-43b5-a30c-a73c111c98d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6ae9a28746242afa489fc8650c2c632, user_id=618618df80e846449ab61d7daa875bfb, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:46.203530 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec69fa-1356-43b5-a30c-a73c111c98d7 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:46.222696 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1374/2731] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:01:46 2022] PUT /identity/v3/projects/f6ae9a28746242afa489fc8650c2c632/users/618618df80e846449ab61d7daa875bfb/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:46.227655 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ffd92b-9565-4acb-b0da-cf772e37b8f1 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:46.239313 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ffd92b-9565-4acb-b0da-cf772e37b8f1 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:46.316756 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d37bf5-ad26-4264-a7a3-45727e2cf7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:46.317664 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d37bf5-ad26-4264-a7a3-45727e2cf7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:46.318549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d37bf5-ad26-4264-a7a3-45727e2cf7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:46.318549 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d37bf5-ad26-4264-a7a3-45727e2cf7a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:46.319559 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d37bf5-ad26-4264-a7a3-45727e2cf7a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:46.321901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d37bf5-ad26-4264-a7a3-45727e2cf7a9 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:46.333509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1358/2732] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:46.338054 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec69fa-1356-43b5-a30c-a73c111c98d7 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:46.351257 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec69fa-1356-43b5-a30c-a73c111c98d7 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:46.358716 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a217894-7ba2-4cda-b340-a35622ee2d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:46.359608 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a217894-7ba2-4cda-b340-a35622ee2d33 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:46.359998 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a217894-7ba2-4cda-b340-a35622ee2d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:46.359998 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a217894-7ba2-4cda-b340-a35622ee2d33 admin admin] PATH_INFO: `/v3/projects/f6ae9a28746242afa489fc8650c2c632/users/618618df80e846449ab61d7daa875bfb/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:46.367286 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a217894-7ba2-4cda-b340-a35622ee2d33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6ae9a28746242afa489fc8650c2c632, user_id=618618df80e846449ab61d7daa875bfb, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:46.369592 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a217894-7ba2-4cda-b340-a35622ee2d33 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:46.386930 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1375/2733] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:01:46 2022] PUT /identity/v3/projects/f6ae9a28746242afa489fc8650c2c632/users/618618df80e846449ab61d7daa875bfb/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:47.488851 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46c32d-3857-408f-b4b9-4d2ed6c2d650 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:47.489688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46c32d-3857-408f-b4b9-4d2ed6c2d650 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:47.489688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46c32d-3857-408f-b4b9-4d2ed6c2d650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:47.544249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-fa46c32d-3857-408f-b4b9-4d2ed6c2d650 None None] MFA Rules not processed for user `618618df80e846449ab61d7daa875bfb`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:47.545945 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-fa46c32d-3857-408f-b4b9-4d2ed6c2d650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:47.743338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1359/2734] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:47 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:49.392103 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a217894-7ba2-4cda-b340-a35622ee2d33 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:49.471877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a217894-7ba2-4cda-b340-a35622ee2d33 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:49.477136 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab37fe-032f-4f22-9687-8b4368be7ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:49.477774 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab37fe-032f-4f22-9687-8b4368be7ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:49.478050 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab37fe-032f-4f22-9687-8b4368be7ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:49.478323 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab37fe-032f-4f22-9687-8b4368be7ea3 admin admin] PATH_INFO: `/v3/users/23c56ba26a0946cd9aff74c8bdf66582` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:49.479475 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab37fe-032f-4f22-9687-8b4368be7ea3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23c56ba26a0946cd9aff74c8bdf66582)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:49.481563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab37fe-032f-4f22-9687-8b4368be7ea3 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:49.544723 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-71ab37fe-032f-4f22-9687-8b4368be7ea3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23c56ba26a0946cd9aff74c8bdf66582'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:49.563131 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-71ab37fe-032f-4f22-9687-8b4368be7ea3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23c56ba26a0946cd9aff74c8bdf66582'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:49.573905 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-71ab37fe-032f-4f22-9687-8b4368be7ea3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23c56ba26a0946cd9aff74c8bdf66582'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:49.583105 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-71ab37fe-032f-4f22-9687-8b4368be7ea3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23c56ba26a0946cd9aff74c8bdf66582'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:49.583874 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-71ab37fe-032f-4f22-9687-8b4368be7ea3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23c56ba26a0946cd9aff74c8bdf66582'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:49.591267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1376/2735] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:01:49 2022] DELETE /identity/v3/users/23c56ba26a0946cd9aff74c8bdf66582 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:49.795947 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa46c32d-3857-408f-b4b9-4d2ed6c2d650 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:49.807839 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa46c32d-3857-408f-b4b9-4d2ed6c2d650 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:49.871668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:49.880304 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56262da-3fef-40ff-b8e4-a0ae8699e816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:49.881269 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56262da-3fef-40ff-b8e4-a0ae8699e816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:49.881595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56262da-3fef-40ff-b8e4-a0ae8699e816 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:49.881959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56262da-3fef-40ff-b8e4-a0ae8699e816 admin admin] PATH_INFO: `/v3/projects/e5e1b236c287477fad700d5343b2a8a2` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:49.883409 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56262da-3fef-40ff-b8e4-a0ae8699e816 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5e1b236c287477fad700d5343b2a8a2)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:49.885467 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56262da-3fef-40ff-b8e4-a0ae8699e816 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:49.947365 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e56262da-3fef-40ff-b8e4-a0ae8699e816 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5e1b236c287477fad700d5343b2a8a2', 'request_id': 'req-e56262da-3fef-40ff-b8e4-a0ae8699e816'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:49.964830 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e56262da-3fef-40ff-b8e4-a0ae8699e816 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5e1b236c287477fad700d5343b2a8a2', 'request_id': 'req-e56262da-3fef-40ff-b8e4-a0ae8699e816'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:49.973327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e56262da-3fef-40ff-b8e4-a0ae8699e816 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5e1b236c287477fad700d5343b2a8a2', 'request_id': 'req-e56262da-3fef-40ff-b8e4-a0ae8699e816'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:49.985679 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e56262da-3fef-40ff-b8e4-a0ae8699e816 admin admin] The token cache is being invalidate because project e5e1b236c287477fad700d5343b2a8a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:01:49.985974 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e56262da-3fef-40ff-b8e4-a0ae8699e816 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:49.986676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e56262da-3fef-40ff-b8e4-a0ae8699e816 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:49.988747 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1360/2736] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:01:49 2022] DELETE /identity/v3/projects/e5e1b236c287477fad700d5343b2a8a2 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:49.993071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab37fe-032f-4f22-9687-8b4368be7ea3 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:49.994732 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-71ab37fe-032f-4f22-9687-8b4368be7ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:50.011210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab37fe-032f-4f22-9687-8b4368be7ea3 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:50.026112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:50.082955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46c91f-722a-4cae-8fe7-5377bb0b0b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:50.082955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46c91f-722a-4cae-8fe7-5377bb0b0b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:50.082955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46c91f-722a-4cae-8fe7-5377bb0b0b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:50.082955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46c91f-722a-4cae-8fe7-5377bb0b0b80 admin admin] PATH_INFO: `/v3/users/618618df80e846449ab61d7daa875bfb` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:50.086078 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46c91f-722a-4cae-8fe7-5377bb0b0b80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=618618df80e846449ab61d7daa875bfb)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:50.086078 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46c91f-722a-4cae-8fe7-5377bb0b0b80 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:50.140612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6b46c91f-722a-4cae-8fe7-5377bb0b0b80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '618618df80e846449ab61d7daa875bfb'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:50.153407 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6b46c91f-722a-4cae-8fe7-5377bb0b0b80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '618618df80e846449ab61d7daa875bfb'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:50.162860 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6b46c91f-722a-4cae-8fe7-5377bb0b0b80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '618618df80e846449ab61d7daa875bfb'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:50.171842 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6b46c91f-722a-4cae-8fe7-5377bb0b0b80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '618618df80e846449ab61d7daa875bfb'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:50.172561 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6b46c91f-722a-4cae-8fe7-5377bb0b0b80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '618618df80e846449ab61d7daa875bfb'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:50.179320 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1377/2737] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:01:49 2022] DELETE /identity/v3/users/618618df80e846449ab61d7daa875bfb => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:50.392037 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56262da-3fef-40ff-b8e4-a0ae8699e816 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:50.393721 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e56262da-3fef-40ff-b8e4-a0ae8699e816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:50.408554 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56262da-3fef-40ff-b8e4-a0ae8699e816 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:50.484771 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbdf6c5-f259-4fe7-bc4d-34b37cd4908d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:50.485361 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdf6c5-f259-4fe7-bc4d-34b37cd4908d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:50.485675 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdf6c5-f259-4fe7-bc4d-34b37cd4908d admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:50.485936 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdf6c5-f259-4fe7-bc4d-34b37cd4908d admin admin] PATH_INFO: `/v3/projects/f6ae9a28746242afa489fc8650c2c632` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:50.487487 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbdf6c5-f259-4fe7-bc4d-34b37cd4908d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6ae9a28746242afa489fc8650c2c632)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:50.489504 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbdf6c5-f259-4fe7-bc4d-34b37cd4908d admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:50.547216 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1bbdf6c5-f259-4fe7-bc4d-34b37cd4908d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6ae9a28746242afa489fc8650c2c632', 'request_id': 'req-1bbdf6c5-f259-4fe7-bc4d-34b37cd4908d'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:50.556443 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1bbdf6c5-f259-4fe7-bc4d-34b37cd4908d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6ae9a28746242afa489fc8650c2c632', 'request_id': 'req-1bbdf6c5-f259-4fe7-bc4d-34b37cd4908d'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:50.563833 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1bbdf6c5-f259-4fe7-bc4d-34b37cd4908d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6ae9a28746242afa489fc8650c2c632', 'request_id': 'req-1bbdf6c5-f259-4fe7-bc4d-34b37cd4908d'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:50.578706 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1bbdf6c5-f259-4fe7-bc4d-34b37cd4908d admin admin] The token cache is being invalidate because project f6ae9a28746242afa489fc8650c2c632 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:01:50.579053 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1bbdf6c5-f259-4fe7-bc4d-34b37cd4908d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:50.579856 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-1bbdf6c5-f259-4fe7-bc4d-34b37cd4908d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:01:50.581876 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1361/2738] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:01:50 2022] DELETE /identity/v3/projects/f6ae9a28746242afa489fc8650c2c632 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:50.588426 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733c774-e91b-45db-a211-7a8a8f09987b None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:50.589099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733c774-e91b-45db-a211-7a8a8f09987b None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:50.589367 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733c774-e91b-45db-a211-7a8a8f09987b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:50.614052 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-f733c774-e91b-45db-a211-7a8a8f09987b None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:50.615362 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f733c774-e91b-45db-a211-7a8a8f09987b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:50.689755 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1378/2739] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:50.695923 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165f9d0-6aae-472e-a53a-3557a8a09d7a None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:50.696190 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165f9d0-6aae-472e-a53a-3557a8a09d7a None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:50.696459 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165f9d0-6aae-472e-a53a-3557a8a09d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:50.720151 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b165f9d0-6aae-472e-a53a-3557a8a09d7a None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:50.721531 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b165f9d0-6aae-472e-a53a-3557a8a09d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:50.732018 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1362/2740] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:50.736631 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733c774-e91b-45db-a211-7a8a8f09987b None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:50.747399 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733c774-e91b-45db-a211-7a8a8f09987b None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:50.753666 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82a440-37af-4906-99de-2c4e5e30abcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:50.754345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82a440-37af-4906-99de-2c4e5e30abcd admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:50.754639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82a440-37af-4906-99de-2c4e5e30abcd admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:50.754876 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae82a440-37af-4906-99de-2c4e5e30abcd admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:50.755627 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae82a440-37af-4906-99de-2c4e5e30abcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 09:01:50.755932 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae82a440-37af-4906-99de-2c4e5e30abcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:50.757955 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae82a440-37af-4906-99de-2c4e5e30abcd admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:50.770817 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1379/2741] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 09:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 09:01:50.776119 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165f9d0-6aae-472e-a53a-3557a8a09d7a None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:50.787861 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165f9d0-6aae-472e-a53a-3557a8a09d7a None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:50.792798 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9a019-13c3-4e7f-bde7-f526eae1f15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:50.793403 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9a019-13c3-4e7f-bde7-f526eae1f15a admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:50.793676 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9a019-13c3-4e7f-bde7-f526eae1f15a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:50.793968 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9a019-13c3-4e7f-bde7-f526eae1f15a admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:50.794590 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f9a019-13c3-4e7f-bde7-f526eae1f15a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:50.797610 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f9a019-13c3-4e7f-bde7-f526eae1f15a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:50.810525 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:50.844582 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1363/2742] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:01:50 2022] POST /identity/v3/projects => generated 371 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:01:50.849555 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82a440-37af-4906-99de-2c4e5e30abcd admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:50.860364 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae82a440-37af-4906-99de-2c4e5e30abcd admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:50.883566 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:50.928526 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1100cd1e-3408-482f-838b-da1aca504cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:50.929150 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100cd1e-3408-482f-838b-da1aca504cf2 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:50.929423 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100cd1e-3408-482f-838b-da1aca504cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:50.929693 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100cd1e-3408-482f-838b-da1aca504cf2 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:50.930348 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1100cd1e-3408-482f-838b-da1aca504cf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:50.932284 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1100cd1e-3408-482f-838b-da1aca504cf2 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:50.962048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1380/2743] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:01:50 2022] POST /identity/v3/users => generated 286 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 09:01:50.966460 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9a019-13c3-4e7f-bde7-f526eae1f15a admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:50.978285 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9a019-13c3-4e7f-bde7-f526eae1f15a admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:50.984233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c517b-b233-47ae-89a6-2d4f6d50c47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:50.985017 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c517b-b233-47ae-89a6-2d4f6d50c47f admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:50.985430 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c517b-b233-47ae-89a6-2d4f6d50c47f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:50.985812 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c517b-b233-47ae-89a6-2d4f6d50c47f admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:50.986613 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c517b-b233-47ae-89a6-2d4f6d50c47f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:50.988734 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c517b-b233-47ae-89a6-2d4f6d50c47f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:51.000764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1364/2744] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:51.006710 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1100cd1e-3408-482f-838b-da1aca504cf2 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:51.019467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1100cd1e-3408-482f-838b-da1aca504cf2 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:51.027276 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb3f83-1507-4b6f-ab8a-cf0c5b35e22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:51.028008 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb3f83-1507-4b6f-ab8a-cf0c5b35e22a admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:51.028286 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb3f83-1507-4b6f-ab8a-cf0c5b35e22a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:51.028552 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb3f83-1507-4b6f-ab8a-cf0c5b35e22a admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:51.029218 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb3f83-1507-4b6f-ab8a-cf0c5b35e22a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:51.030836 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb3f83-1507-4b6f-ab8a-cf0c5b35e22a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:51.043371 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1381/2745] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:51.049232 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c517b-b233-47ae-89a6-2d4f6d50c47f admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:51.061994 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c517b-b233-47ae-89a6-2d4f6d50c47f admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:51.068472 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44313b85-35ba-425d-9d77-400c72153f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:51.069206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44313b85-35ba-425d-9d77-400c72153f4e admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:51.071129 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44313b85-35ba-425d-9d77-400c72153f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:51.071413 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44313b85-35ba-425d-9d77-400c72153f4e admin admin] PATH_INFO: `/v3/projects/fc73b5e9132b482798c0c359a4a5d9be/users/8340ffdcb82f466e99897d99b0908e91/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:51.090405 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44313b85-35ba-425d-9d77-400c72153f4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc73b5e9132b482798c0c359a4a5d9be, user_id=8340ffdcb82f466e99897d99b0908e91, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:51.093137 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44313b85-35ba-425d-9d77-400c72153f4e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:51.109888 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1365/2746] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:01:51 2022] PUT /identity/v3/projects/fc73b5e9132b482798c0c359a4a5d9be/users/8340ffdcb82f466e99897d99b0908e91/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:51.122229 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb79b82-b8cc-4283-bc18-599399fd1649 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:51.122555 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb79b82-b8cc-4283-bc18-599399fd1649 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:51.122801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb79b82-b8cc-4283-bc18-599399fd1649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:51.153476 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-bcb79b82-b8cc-4283-bc18-599399fd1649 None None] MFA Rules not processed for user `8340ffdcb82f466e99897d99b0908e91`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:51.154787 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-bcb79b82-b8cc-4283-bc18-599399fd1649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:51.235187 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1382/2747] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:51.242152 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44313b85-35ba-425d-9d77-400c72153f4e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:51.314968 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44313b85-35ba-425d-9d77-400c72153f4e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:51.319617 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c82f87-447c-4d24-b728-4b13b9da026a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:51.320229 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c82f87-447c-4d24-b728-4b13b9da026a admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:51.320501 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c82f87-447c-4d24-b728-4b13b9da026a admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:51.320766 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c82f87-447c-4d24-b728-4b13b9da026a admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:51.321382 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c82f87-447c-4d24-b728-4b13b9da026a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:51.323440 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c82f87-447c-4d24-b728-4b13b9da026a admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:51.354566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1366/2748] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:01:51 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:01:51.359309 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb79b82-b8cc-4283-bc18-599399fd1649 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:51.370177 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb79b82-b8cc-4283-bc18-599399fd1649 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:51.439855 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b4bd0-64ec-4360-8d1b-6d8448eb9dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:51.440542 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b4bd0-64ec-4360-8d1b-6d8448eb9dba admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:51.440962 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b4bd0-64ec-4360-8d1b-6d8448eb9dba admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:51.441161 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b4bd0-64ec-4360-8d1b-6d8448eb9dba admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:51.441897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b4bd0-64ec-4360-8d1b-6d8448eb9dba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:51.443906 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b4bd0-64ec-4360-8d1b-6d8448eb9dba admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:51.470122 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1383/2749] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:01:51 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:01:51.474735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c82f87-447c-4d24-b728-4b13b9da026a admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:51.486392 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c82f87-447c-4d24-b728-4b13b9da026a admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:51.491289 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5239b-4276-4060-a17c-64238987c12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:51.491888 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5239b-4276-4060-a17c-64238987c12e admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:51.492220 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5239b-4276-4060-a17c-64238987c12e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:51.492606 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5239b-4276-4060-a17c-64238987c12e admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:51.493188 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5239b-4276-4060-a17c-64238987c12e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:51.494922 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5239b-4276-4060-a17c-64238987c12e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:51.511340 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1367/2750] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:51.517250 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b4bd0-64ec-4360-8d1b-6d8448eb9dba admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:51.528612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b4bd0-64ec-4360-8d1b-6d8448eb9dba admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:51.534922 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba004dbc-098a-488f-ab81-167ce7db1da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:51.535602 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba004dbc-098a-488f-ab81-167ce7db1da8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:51.535854 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba004dbc-098a-488f-ab81-167ce7db1da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:51.536165 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba004dbc-098a-488f-ab81-167ce7db1da8 admin admin] PATH_INFO: `/v3/projects/dcaaf344e67c46829646fcc09361aed8/users/1785bb2d8dd54b0eb0d8c4b0312a141b/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:51.557590 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba004dbc-098a-488f-ab81-167ce7db1da8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcaaf344e67c46829646fcc09361aed8, user_id=1785bb2d8dd54b0eb0d8c4b0312a141b, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:51.560429 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba004dbc-098a-488f-ab81-167ce7db1da8 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:51.571987 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:51.580450 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1384/2751] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:01:51 2022] PUT /identity/v3/projects/dcaaf344e67c46829646fcc09361aed8/users/1785bb2d8dd54b0eb0d8c4b0312a141b/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 09:01:51.585623 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5239b-4276-4060-a17c-64238987c12e admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:51.599597 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5239b-4276-4060-a17c-64238987c12e admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:51.670103 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14678d6e-4696-4527-bcc2-bb7d07858d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:51.670719 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14678d6e-4696-4527-bcc2-bb7d07858d68 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:51.671036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14678d6e-4696-4527-bcc2-bb7d07858d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:51.671313 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14678d6e-4696-4527-bcc2-bb7d07858d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:51.671969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14678d6e-4696-4527-bcc2-bb7d07858d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:51.673606 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14678d6e-4696-4527-bcc2-bb7d07858d68 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:51.684468 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1368/2752] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:01:51.689426 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba004dbc-098a-488f-ab81-167ce7db1da8 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:51.702658 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba004dbc-098a-488f-ab81-167ce7db1da8 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:51.708872 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9cd411-6f8b-4198-bb6a-ac3bf68bce59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:51.709589 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cd411-6f8b-4198-bb6a-ac3bf68bce59 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:51.710202 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cd411-6f8b-4198-bb6a-ac3bf68bce59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:51.710202 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cd411-6f8b-4198-bb6a-ac3bf68bce59 admin admin] PATH_INFO: `/v3/projects/dcaaf344e67c46829646fcc09361aed8/users/1785bb2d8dd54b0eb0d8c4b0312a141b/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:51.712824 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9cd411-6f8b-4198-bb6a-ac3bf68bce59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcaaf344e67c46829646fcc09361aed8, user_id=1785bb2d8dd54b0eb0d8c4b0312a141b, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:51.715800 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9cd411-6f8b-4198-bb6a-ac3bf68bce59 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:51.727960 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1385/2753] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:01:51 2022] PUT /identity/v3/projects/dcaaf344e67c46829646fcc09361aed8/users/1785bb2d8dd54b0eb0d8c4b0312a141b/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:01:51.740782 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574862d-f9ef-433d-a4f2-a746f44da6d2 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:51.741233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574862d-f9ef-433d-a4f2-a746f44da6d2 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:51.741581 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574862d-f9ef-433d-a4f2-a746f44da6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:51.787280 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-a574862d-f9ef-433d-a4f2-a746f44da6d2 None None] MFA Rules not processed for user `1785bb2d8dd54b0eb0d8c4b0312a141b`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:01:51.788407 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a574862d-f9ef-433d-a4f2-a746f44da6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:51.800978 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:01:51.874181 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1369/2754] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:01:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:01:51.886851 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9cd411-6f8b-4198-bb6a-ac3bf68bce59 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:51.889410 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ff9cd411-6f8b-4198-bb6a-ac3bf68bce59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:01:51.979710 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9cd411-6f8b-4198-bb6a-ac3bf68bce59 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:51.983425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ced897-a43a-4afc-b14f-6474c56314dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:51.984047 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ced897-a43a-4afc-b14f-6474c56314dc service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:51.984326 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ced897-a43a-4afc-b14f-6474c56314dc service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:51.984602 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ced897-a43a-4afc-b14f-6474c56314dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:51.995436 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ced897-a43a-4afc-b14f-6474c56314dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:51.996911 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ced897-a43a-4afc-b14f-6474c56314dc service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:52.017994 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1386/2755] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:01:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:01:52.103187 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574862d-f9ef-433d-a4f2-a746f44da6d2 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:01:52.114233 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574862d-f9ef-433d-a4f2-a746f44da6d2 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:01:52.118403 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d376ff-444b-462e-9969-0bdebec391bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:01:52.119059 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d376ff-444b-462e-9969-0bdebec391bc service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:01:52.119338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d376ff-444b-462e-9969-0bdebec391bc service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:01:52.119610 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d376ff-444b-462e-9969-0bdebec391bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:01:52.189700 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d376ff-444b-462e-9969-0bdebec391bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:01:52.191650 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d376ff-444b-462e-9969-0bdebec391bc service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:01:52.202093 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1370/2756] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:01:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:02:02.003046 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=32,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:02:02.197700 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=51 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:02:16.625112 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ced897-a43a-4afc-b14f-6474c56314dc service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:16.626733 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b8ced897-a43a-4afc-b14f-6474c56314dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:16.714740 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ced897-a43a-4afc-b14f-6474c56314dc service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:16.720917 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1988a6-e696-45e6-bda7-9f20d90af095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:16.721577 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1988a6-e696-45e6-bda7-9f20d90af095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:16.721854 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1988a6-e696-45e6-bda7-9f20d90af095 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:16.722174 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1988a6-e696-45e6-bda7-9f20d90af095 admin admin] PATH_INFO: `/v3/users/14b7a7358617411ea98023f054139bbc` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:16.723361 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1988a6-e696-45e6-bda7-9f20d90af095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14b7a7358617411ea98023f054139bbc)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:16.725336 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1988a6-e696-45e6-bda7-9f20d90af095 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:16.791539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6b1988a6-e696-45e6-bda7-9f20d90af095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14b7a7358617411ea98023f054139bbc'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:16.805309 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6b1988a6-e696-45e6-bda7-9f20d90af095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14b7a7358617411ea98023f054139bbc'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:16.815814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6b1988a6-e696-45e6-bda7-9f20d90af095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14b7a7358617411ea98023f054139bbc'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:16.828475 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6b1988a6-e696-45e6-bda7-9f20d90af095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14b7a7358617411ea98023f054139bbc'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:16.829614 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-6b1988a6-e696-45e6-bda7-9f20d90af095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14b7a7358617411ea98023f054139bbc'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:16.840642 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1387/2757] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:02:16 2022] DELETE /identity/v3/users/14b7a7358617411ea98023f054139bbc => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 09:02:16.846223 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2653a-7382-4770-b24e-06909b5b4878 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:16.846959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2653a-7382-4770-b24e-06909b5b4878 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:16.846959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2653a-7382-4770-b24e-06909b5b4878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:16.869883 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-b4e2653a-7382-4770-b24e-06909b5b4878 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:02:16.871152 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b4e2653a-7382-4770-b24e-06909b5b4878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:16.948548 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1371/2758] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:02:16.960242 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1988a6-e696-45e6-bda7-9f20d90af095 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:16.962041 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-6b1988a6-e696-45e6-bda7-9f20d90af095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:17.038174 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1988a6-e696-45e6-bda7-9f20d90af095 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:17.042177 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0218e-5abd-4133-bd31-ed0f6f5b7e26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:17.042907 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0218e-5abd-4133-bd31-ed0f6f5b7e26 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:17.043300 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0218e-5abd-4133-bd31-ed0f6f5b7e26 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:17.043444 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0218e-5abd-4133-bd31-ed0f6f5b7e26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:17.060987 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0218e-5abd-4133-bd31-ed0f6f5b7e26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:17.062722 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0218e-5abd-4133-bd31-ed0f6f5b7e26 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:17.074588 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1388/2759] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:02:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:02:17.426838 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2653a-7382-4770-b24e-06909b5b4878 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:17.428476 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-b4e2653a-7382-4770-b24e-06909b5b4878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:17.443602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2653a-7382-4770-b24e-06909b5b4878 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:17.451835 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477046c-3c75-427a-be7e-184c340e0653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:17.453066 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477046c-3c75-427a-be7e-184c340e0653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:17.463240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477046c-3c75-427a-be7e-184c340e0653 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:17.463240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477046c-3c75-427a-be7e-184c340e0653 admin admin] PATH_INFO: `/v3/projects/683f469c5bf74200b35f846c4ad64d14` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:17.463240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4477046c-3c75-427a-be7e-184c340e0653 admin admin] RBAC: Authorizing `identity:delete_project(project_id=683f469c5bf74200b35f846c4ad64d14)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:17.465141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4477046c-3c75-427a-be7e-184c340e0653 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:17.539532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4477046c-3c75-427a-be7e-184c340e0653 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '683f469c5bf74200b35f846c4ad64d14', 'request_id': 'req-4477046c-3c75-427a-be7e-184c340e0653'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:17.550225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4477046c-3c75-427a-be7e-184c340e0653 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '683f469c5bf74200b35f846c4ad64d14', 'request_id': 'req-4477046c-3c75-427a-be7e-184c340e0653'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:17.557286 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4477046c-3c75-427a-be7e-184c340e0653 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '683f469c5bf74200b35f846c4ad64d14', 'request_id': 'req-4477046c-3c75-427a-be7e-184c340e0653'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:17.571988 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4477046c-3c75-427a-be7e-184c340e0653 admin admin] The token cache is being invalidate because project 683f469c5bf74200b35f846c4ad64d14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:02:17.572287 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4477046c-3c75-427a-be7e-184c340e0653 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:17.573061 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-4477046c-3c75-427a-be7e-184c340e0653 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:17.574550 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1372/2760] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:02:17 2022] DELETE /identity/v3/projects/683f469c5bf74200b35f846c4ad64d14 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:02:17.582402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab6bc7-5445-40fe-b98b-c9972353a80a None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:17.582695 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab6bc7-5445-40fe-b98b-c9972353a80a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:17.583097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab6bc7-5445-40fe-b98b-c9972353a80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:17.604904 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-36ab6bc7-5445-40fe-b98b-c9972353a80a None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:02:17.606271 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-36ab6bc7-5445-40fe-b98b-c9972353a80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:17.631582 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:02:17.706290 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1389/2761] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:02:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:02:17.775869 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c46465-38cc-470f-864c-8a678d7386ce None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:17.775869 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c46465-38cc-470f-864c-8a678d7386ce None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:17.775869 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c46465-38cc-470f-864c-8a678d7386ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:17.809545 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-a8c46465-38cc-470f-864c-8a678d7386ce None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:02:17.810777 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a8c46465-38cc-470f-864c-8a678d7386ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:17.831517 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1373/2762] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:02:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:02:17.831693 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab6bc7-5445-40fe-b98b-c9972353a80a None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:17.838371 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab6bc7-5445-40fe-b98b-c9972353a80a None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:17.843610 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e0fb8-a522-4526-9208-2111a4740fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:17.843912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e0fb8-a522-4526-9208-2111a4740fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:17.844201 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e0fb8-a522-4526-9208-2111a4740fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:17.844474 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e0fb8-a522-4526-9208-2111a4740fe8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:17.845141 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e0fb8-a522-4526-9208-2111a4740fe8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 09:02:17.845439 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e0fb8-a522-4526-9208-2111a4740fe8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:17.847154 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e0fb8-a522-4526-9208-2111a4740fe8 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:17.865283 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1390/2763] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 09:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 09:02:17.870024 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c46465-38cc-470f-864c-8a678d7386ce None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:17.883212 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c46465-38cc-470f-864c-8a678d7386ce None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:17.889027 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0823f3-6dcf-46aa-a95e-67a88351376c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:17.889656 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0823f3-6dcf-46aa-a95e-67a88351376c admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:17.889925 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0823f3-6dcf-46aa-a95e-67a88351376c admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:17.890194 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0823f3-6dcf-46aa-a95e-67a88351376c admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:17.890843 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0823f3-6dcf-46aa-a95e-67a88351376c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:17.892831 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0823f3-6dcf-46aa-a95e-67a88351376c admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:17.925943 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1374/2764] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:02:17 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 09:02:17.930649 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e0fb8-a522-4526-9208-2111a4740fe8 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:17.941216 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e0fb8-a522-4526-9208-2111a4740fe8 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:18.014712 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060f7eb-1fb4-416b-acb8-a5570f70f579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:18.015331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060f7eb-1fb4-416b-acb8-a5570f70f579 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:18.015729 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060f7eb-1fb4-416b-acb8-a5570f70f579 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:18.015913 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060f7eb-1fb4-416b-acb8-a5570f70f579 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:18.016543 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6060f7eb-1fb4-416b-acb8-a5570f70f579 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:18.018487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6060f7eb-1fb4-416b-acb8-a5570f70f579 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:18.046429 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1391/2765] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:02:17 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:02:18.052498 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0823f3-6dcf-46aa-a95e-67a88351376c admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:18.064014 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0823f3-6dcf-46aa-a95e-67a88351376c admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:18.068783 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62571eb9-a8cc-4be9-a552-095480c36dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:18.069423 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62571eb9-a8cc-4be9-a552-095480c36dfa admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:18.069706 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62571eb9-a8cc-4be9-a552-095480c36dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:18.069981 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62571eb9-a8cc-4be9-a552-095480c36dfa admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:18.070667 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62571eb9-a8cc-4be9-a552-095480c36dfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:18.072351 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62571eb9-a8cc-4be9-a552-095480c36dfa admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:18.089412 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1375/2766] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 09:02:18.096352 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060f7eb-1fb4-416b-acb8-a5570f70f579 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:18.112612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060f7eb-1fb4-416b-acb8-a5570f70f579 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:18.123415 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67524b04-6b0b-4479-9700-9c5f7decf63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:18.124419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67524b04-6b0b-4479-9700-9c5f7decf63e admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:18.124721 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67524b04-6b0b-4479-9700-9c5f7decf63e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:18.125016 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67524b04-6b0b-4479-9700-9c5f7decf63e admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:18.125812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67524b04-6b0b-4479-9700-9c5f7decf63e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:18.127803 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67524b04-6b0b-4479-9700-9c5f7decf63e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:18.143266 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1392/2767] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:02:18.149311 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62571eb9-a8cc-4be9-a552-095480c36dfa admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:18.164359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62571eb9-a8cc-4be9-a552-095480c36dfa admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:18.175075 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e30049-1f58-4069-af49-81d9be48ccc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:18.175965 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e30049-1f58-4069-af49-81d9be48ccc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:18.176444 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e30049-1f58-4069-af49-81d9be48ccc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:18.176968 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e30049-1f58-4069-af49-81d9be48ccc5 admin admin] PATH_INFO: `/v3/projects/3f752219ef5443a08876f6dad7c024bd/users/37ac338045714e3e81ecb9a62ab0e28e/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:18.200741 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e30049-1f58-4069-af49-81d9be48ccc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f752219ef5443a08876f6dad7c024bd, user_id=37ac338045714e3e81ecb9a62ab0e28e, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:18.203550 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e30049-1f58-4069-af49-81d9be48ccc5 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:18.227794 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1376/2768] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:02:18 2022] PUT /identity/v3/projects/3f752219ef5443a08876f6dad7c024bd/users/37ac338045714e3e81ecb9a62ab0e28e/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:02:18.242563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d681d-cd1c-47c0-9073-a3d2fff0775a None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:18.243071 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d681d-cd1c-47c0-9073-a3d2fff0775a None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:18.243522 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998d681d-cd1c-47c0-9073-a3d2fff0775a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:18.283695 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-998d681d-cd1c-47c0-9073-a3d2fff0775a None None] MFA Rules not processed for user `37ac338045714e3e81ecb9a62ab0e28e`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:02:18.285782 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-998d681d-cd1c-47c0-9073-a3d2fff0775a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:18.408090 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1393/2769] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:02:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:02:18.412096 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e30049-1f58-4069-af49-81d9be48ccc5 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:18.444282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:02:18.505773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e30049-1f58-4069-af49-81d9be48ccc5 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:18.511258 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187b954-fce2-4fa6-af45-3d0df893ea55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:18.512312 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187b954-fce2-4fa6-af45-3d0df893ea55 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:18.512927 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187b954-fce2-4fa6-af45-3d0df893ea55 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:18.513209 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187b954-fce2-4fa6-af45-3d0df893ea55 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:18.513879 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2187b954-fce2-4fa6-af45-3d0df893ea55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:18.515874 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2187b954-fce2-4fa6-af45-3d0df893ea55 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:18.548714 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1377/2770] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:02:18 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:02:18.552748 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d681d-cd1c-47c0-9073-a3d2fff0775a None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:18.572146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998d681d-cd1c-47c0-9073-a3d2fff0775a None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:18.592884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:02:18.654110 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2408087-6b8e-4976-bbc2-f6f40ea24c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:18.655115 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2408087-6b8e-4976-bbc2-f6f40ea24c44 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:18.655266 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2408087-6b8e-4976-bbc2-f6f40ea24c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:18.655411 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2408087-6b8e-4976-bbc2-f6f40ea24c44 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:18.656045 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2408087-6b8e-4976-bbc2-f6f40ea24c44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:18.657986 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2408087-6b8e-4976-bbc2-f6f40ea24c44 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:18.686000 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1394/2771] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:02:18 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:02:18.690352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187b954-fce2-4fa6-af45-3d0df893ea55 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:18.706177 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187b954-fce2-4fa6-af45-3d0df893ea55 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:18.712373 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10461d-c7b7-4614-8f9d-5d10d4d77d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:18.713083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10461d-c7b7-4614-8f9d-5d10d4d77d94 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:18.713426 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10461d-c7b7-4614-8f9d-5d10d4d77d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:18.713792 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd10461d-c7b7-4614-8f9d-5d10d4d77d94 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:18.714536 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd10461d-c7b7-4614-8f9d-5d10d4d77d94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:18.716332 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd10461d-c7b7-4614-8f9d-5d10d4d77d94 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:18.728918 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1378/2772] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:02:18.733887 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2408087-6b8e-4976-bbc2-f6f40ea24c44 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:18.745558 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2408087-6b8e-4976-bbc2-f6f40ea24c44 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:18.751366 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbdb1a6-8ee1-4190-bb21-d0b11b31a192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:18.752000 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdb1a6-8ee1-4190-bb21-d0b11b31a192 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:18.752277 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdb1a6-8ee1-4190-bb21-d0b11b31a192 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:18.752545 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdb1a6-8ee1-4190-bb21-d0b11b31a192 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:18.753202 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbdb1a6-8ee1-4190-bb21-d0b11b31a192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:18.754849 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbdb1a6-8ee1-4190-bb21-d0b11b31a192 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:18.768993 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1395/2773] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:02:18.774491 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10461d-c7b7-4614-8f9d-5d10d4d77d94 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:18.786621 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd10461d-c7b7-4614-8f9d-5d10d4d77d94 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:18.791643 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0708a7-625c-4310-9190-196a826bd8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:18.792360 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0708a7-625c-4310-9190-196a826bd8b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:18.792634 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0708a7-625c-4310-9190-196a826bd8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:18.792904 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0708a7-625c-4310-9190-196a826bd8b4 admin admin] PATH_INFO: `/v3/projects/ea1952483be74f8e87880f5e3ce7369d/users/116e208b0856494fa10ea42276b2b9cb/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:18.817520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0708a7-625c-4310-9190-196a826bd8b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea1952483be74f8e87880f5e3ce7369d, user_id=116e208b0856494fa10ea42276b2b9cb, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:18.825087 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0708a7-625c-4310-9190-196a826bd8b4 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:18.843277 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1379/2774] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:02:18 2022] PUT /identity/v3/projects/ea1952483be74f8e87880f5e3ce7369d/users/116e208b0856494fa10ea42276b2b9cb/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:02:18.855629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770cd365-8692-46b3-96be-d9589d7f968f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:18.855894 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770cd365-8692-46b3-96be-d9589d7f968f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:18.856170 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770cd365-8692-46b3-96be-d9589d7f968f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:18.889639 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-770cd365-8692-46b3-96be-d9589d7f968f None None] MFA Rules not processed for user `116e208b0856494fa10ea42276b2b9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:02:18.890948 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-770cd365-8692-46b3-96be-d9589d7f968f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:18.996831 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1396/2775] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:02:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:02:19.004053 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0708a7-625c-4310-9190-196a826bd8b4 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:19.130224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0708a7-625c-4310-9190-196a826bd8b4 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:19.138922 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f343c56-25ef-4621-a1e9-9a55249e4b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:19.139688 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f343c56-25ef-4621-a1e9-9a55249e4b0c admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:19.140002 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f343c56-25ef-4621-a1e9-9a55249e4b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:19.140228 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f343c56-25ef-4621-a1e9-9a55249e4b0c admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:19.140945 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f343c56-25ef-4621-a1e9-9a55249e4b0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:19.143084 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f343c56-25ef-4621-a1e9-9a55249e4b0c admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:19.211851 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1380/2776] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:02:19 2022] POST /identity/v3/projects => generated 385 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:02:19.216409 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770cd365-8692-46b3-96be-d9589d7f968f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:19.244320 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770cd365-8692-46b3-96be-d9589d7f968f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:19.379884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960dabdf-9af8-4fe2-9bed-870235b0cbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:19.379884 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960dabdf-9af8-4fe2-9bed-870235b0cbc2 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:19.380487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960dabdf-9af8-4fe2-9bed-870235b0cbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:19.380487 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960dabdf-9af8-4fe2-9bed-870235b0cbc2 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:19.381156 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960dabdf-9af8-4fe2-9bed-870235b0cbc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:19.383571 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960dabdf-9af8-4fe2-9bed-870235b0cbc2 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:19.421885 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1397/2777] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:02:19 2022] POST /identity/v3/users => generated 299 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:02:19.428355 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f343c56-25ef-4621-a1e9-9a55249e4b0c admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:19.455370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f343c56-25ef-4621-a1e9-9a55249e4b0c admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:19.463141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ce33f-522a-42b7-9ad3-b7974c96b0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:19.463792 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ce33f-522a-42b7-9ad3-b7974c96b0ba admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:19.464224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ce33f-522a-42b7-9ad3-b7974c96b0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:19.464503 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ce33f-522a-42b7-9ad3-b7974c96b0ba admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:19.465177 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401ce33f-522a-42b7-9ad3-b7974c96b0ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:19.467246 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401ce33f-522a-42b7-9ad3-b7974c96b0ba admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:19.487855 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1381/2778] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:02:19.495621 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960dabdf-9af8-4fe2-9bed-870235b0cbc2 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:19.537816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960dabdf-9af8-4fe2-9bed-870235b0cbc2 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:19.548316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66607d9d-8dfc-471b-bb4a-a3fd21c043e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:19.549602 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66607d9d-8dfc-471b-bb4a-a3fd21c043e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:19.550068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66607d9d-8dfc-471b-bb4a-a3fd21c043e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:19.550488 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66607d9d-8dfc-471b-bb4a-a3fd21c043e1 admin admin] PATH_INFO: `/v3/projects/68fcb05620e04880872c60a67d230ffe/users/14ebf7c1fbf14eeb9456826ff324ae1e/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:19.578075 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:02:19.584626 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66607d9d-8dfc-471b-bb4a-a3fd21c043e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68fcb05620e04880872c60a67d230ffe, user_id=14ebf7c1fbf14eeb9456826ff324ae1e, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:19.589597 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66607d9d-8dfc-471b-bb4a-a3fd21c043e1 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:19.630263 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1398/2779] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:02:19 2022] PUT /identity/v3/projects/68fcb05620e04880872c60a67d230ffe/users/14ebf7c1fbf14eeb9456826ff324ae1e/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:02:19.635618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ce33f-522a-42b7-9ad3-b7974c96b0ba admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:19.653787 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ce33f-522a-42b7-9ad3-b7974c96b0ba admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:19.731402 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:02:19.755608 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36945538-0bd2-44ca-be6a-c053eb5693c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:19.756593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36945538-0bd2-44ca-be6a-c053eb5693c3 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:19.756869 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36945538-0bd2-44ca-be6a-c053eb5693c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:19.757494 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36945538-0bd2-44ca-be6a-c053eb5693c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:19.758165 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36945538-0bd2-44ca-be6a-c053eb5693c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:19.759970 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36945538-0bd2-44ca-be6a-c053eb5693c3 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:19.782530 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1382/2780] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:02:19.791734 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66607d9d-8dfc-471b-bb4a-a3fd21c043e1 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:19.803638 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66607d9d-8dfc-471b-bb4a-a3fd21c043e1 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:19.809473 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662f323-8457-43db-be57-c089d5a3b063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:19.810155 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662f323-8457-43db-be57-c089d5a3b063 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:19.810430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662f323-8457-43db-be57-c089d5a3b063 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:19.810702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c662f323-8457-43db-be57-c089d5a3b063 admin admin] PATH_INFO: `/v3/projects/68fcb05620e04880872c60a67d230ffe/users/14ebf7c1fbf14eeb9456826ff324ae1e/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:19.812913 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c662f323-8457-43db-be57-c089d5a3b063 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68fcb05620e04880872c60a67d230ffe, user_id=14ebf7c1fbf14eeb9456826ff324ae1e, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:19.815665 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c662f323-8457-43db-be57-c089d5a3b063 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:19.832032 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1399/2781] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:02:19 2022] PUT /identity/v3/projects/68fcb05620e04880872c60a67d230ffe/users/14ebf7c1fbf14eeb9456826ff324ae1e/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:02:19.845031 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31732a4f-07d3-4ed6-bc3f-0e9e32335711 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:19.845323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31732a4f-07d3-4ed6-bc3f-0e9e32335711 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:19.845602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31732a4f-07d3-4ed6-bc3f-0e9e32335711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:19.886656 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-31732a4f-07d3-4ed6-bc3f-0e9e32335711 None None] MFA Rules not processed for user `14ebf7c1fbf14eeb9456826ff324ae1e`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:02:19.887959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-31732a4f-07d3-4ed6-bc3f-0e9e32335711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:20.006202 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1383/2782] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:02:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:02:20.022447 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662f323-8457-43db-be57-c089d5a3b063 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:20.024397 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c662f323-8457-43db-be57-c089d5a3b063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:20.094315 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c662f323-8457-43db-be57-c089d5a3b063 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:20.098628 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df31ec5-0f19-4684-b2a4-7189a97c3a65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:20.099296 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df31ec5-0f19-4684-b2a4-7189a97c3a65 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:20.099589 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df31ec5-0f19-4684-b2a4-7189a97c3a65 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:20.099851 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df31ec5-0f19-4684-b2a4-7189a97c3a65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:20.167193 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df31ec5-0f19-4684-b2a4-7189a97c3a65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:20.170029 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df31ec5-0f19-4684-b2a4-7189a97c3a65 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:20.185663 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1400/2783] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:02:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:02:21.875006 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31732a4f-07d3-4ed6-bc3f-0e9e32335711 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:21.876960 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-31732a4f-07d3-4ed6-bc3f-0e9e32335711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:21.890025 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31732a4f-07d3-4ed6-bc3f-0e9e32335711 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:21.893702 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097af45-91f2-4e83-b8af-329e60ed1fca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:21.894551 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097af45-91f2-4e83-b8af-329e60ed1fca service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:21.894820 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097af45-91f2-4e83-b8af-329e60ed1fca service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:21.895119 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097af45-91f2-4e83-b8af-329e60ed1fca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:21.975462 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2097af45-91f2-4e83-b8af-329e60ed1fca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:21.977125 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2097af45-91f2-4e83-b8af-329e60ed1fca service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:21.989669 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1384/2784] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:02:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:02:22.803309 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df31ec5-0f19-4684-b2a4-7189a97c3a65 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:22.814182 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df31ec5-0f19-4684-b2a4-7189a97c3a65 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:22.820218 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7111b6-e635-48ab-8408-92ef0d08730b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:22.820963 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7111b6-e635-48ab-8408-92ef0d08730b service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:22.821282 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7111b6-e635-48ab-8408-92ef0d08730b service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:22.821616 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7111b6-e635-48ab-8408-92ef0d08730b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:22.834775 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7111b6-e635-48ab-8408-92ef0d08730b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:22.836525 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7111b6-e635-48ab-8408-92ef0d08730b service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:22.848176 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1401/2785] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:02:22 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:02:31.983505 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=71 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:02:32.842774 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=62,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:02:37.789257 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b54f37c-bea6-44c4-8394-5e7d1ea04106 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:37.789929 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b54f37c-bea6-44c4-8394-5e7d1ea04106 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:37.790212 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b54f37c-bea6-44c4-8394-5e7d1ea04106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:37.815214 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-4b54f37c-bea6-44c4-8394-5e7d1ea04106 None None] MFA Rules not processed for user `d5e175245f2e48c79d5f82dbaf4f1cf4`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:02:37.816514 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4b54f37c-bea6-44c4-8394-5e7d1ea04106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:37.882560 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1385/2786] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:02:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:02:38.181251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7111b6-e635-48ab-8408-92ef0d08730b service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:38.183505 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-be7111b6-e635-48ab-8408-92ef0d08730b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:38.261284 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7111b6-e635-48ab-8408-92ef0d08730b service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:38.265789 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7dd6c-eefe-45ff-8c88-0daf72136c5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:38.266440 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7dd6c-eefe-45ff-8c88-0daf72136c5b service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:38.266711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7dd6c-eefe-45ff-8c88-0daf72136c5b service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:38.267010 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7dd6c-eefe-45ff-8c88-0daf72136c5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:38.268958 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-07a7dd6c-eefe-45ff-8c88-0daf72136c5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:38.280324 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7dd6c-eefe-45ff-8c88-0daf72136c5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:38.281815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7dd6c-eefe-45ff-8c88-0daf72136c5b service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:38.291908 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1402/2787] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:02:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:02:38.724593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b54f37c-bea6-44c4-8394-5e7d1ea04106 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:38.727408 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-4b54f37c-bea6-44c4-8394-5e7d1ea04106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:38.797736 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7dd6c-eefe-45ff-8c88-0daf72136c5b service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:38.851393 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b54f37c-bea6-44c4-8394-5e7d1ea04106 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:38.859116 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51c4f2-5bc7-4479-a8df-210fa27cf23f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf5e2b4f5e440a7942adea3d51cec75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:38.860788 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51c4f2-5bc7-4479-a8df-210fa27cf23f service placement] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:38.861220 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51c4f2-5bc7-4479-a8df-210fa27cf23f service placement] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:38.861824 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51c4f2-5bc7-4479-a8df-210fa27cf23f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:38.864297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-aa51c4f2-5bc7-4479-a8df-210fa27cf23f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:38.879755 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa51c4f2-5bc7-4479-a8df-210fa27cf23f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:38.882217 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa51c4f2-5bc7-4479-a8df-210fa27cf23f service placement] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:38.894364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1386/2788] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:02:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:02:38.906425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7dd6c-eefe-45ff-8c88-0daf72136c5b service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:38.916346 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff34596-dd9a-4e58-a087-1eaa47fd16a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:38.917702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff34596-dd9a-4e58-a087-1eaa47fd16a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:38.918448 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff34596-dd9a-4e58-a087-1eaa47fd16a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:38.919051 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff34596-dd9a-4e58-a087-1eaa47fd16a2 admin admin] PATH_INFO: `/v3/users/37ac338045714e3e81ecb9a62ab0e28e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:38.921861 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff34596-dd9a-4e58-a087-1eaa47fd16a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37ac338045714e3e81ecb9a62ab0e28e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:38.924706 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff34596-dd9a-4e58-a087-1eaa47fd16a2 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:39.006501 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-1ff34596-dd9a-4e58-a087-1eaa47fd16a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37ac338045714e3e81ecb9a62ab0e28e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:39.031535 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-1ff34596-dd9a-4e58-a087-1eaa47fd16a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37ac338045714e3e81ecb9a62ab0e28e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:39.048432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-1ff34596-dd9a-4e58-a087-1eaa47fd16a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37ac338045714e3e81ecb9a62ab0e28e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:39.066319 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-1ff34596-dd9a-4e58-a087-1eaa47fd16a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37ac338045714e3e81ecb9a62ab0e28e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:39.067396 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-1ff34596-dd9a-4e58-a087-1eaa47fd16a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37ac338045714e3e81ecb9a62ab0e28e'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:39.079633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1403/2789] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:02:38 2022] DELETE /identity/v3/users/37ac338045714e3e81ecb9a62ab0e28e => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:02:39.085293 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c5eae-db86-4434-9373-ca49aad2ea7b None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:39.085662 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c5eae-db86-4434-9373-ca49aad2ea7b None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:39.086051 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c5eae-db86-4434-9373-ca49aad2ea7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:39.113760 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-c47c5eae-db86-4434-9373-ca49aad2ea7b None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:02:39.115279 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c47c5eae-db86-4434-9373-ca49aad2ea7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:39.218824 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1387/2790] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:02:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:02:39.227809 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff34596-dd9a-4e58-a087-1eaa47fd16a2 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:39.231215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-1ff34596-dd9a-4e58-a087-1eaa47fd16a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:39.269286 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:02:39.316319 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff34596-dd9a-4e58-a087-1eaa47fd16a2 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:39.320496 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab0abe1-70ad-4ca3-aa42-26d2d41eb7af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:39.321119 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0abe1-70ad-4ca3-aa42-26d2d41eb7af service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:39.321387 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0abe1-70ad-4ca3-aa42-26d2d41eb7af service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:39.321660 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0abe1-70ad-4ca3-aa42-26d2d41eb7af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:39.333417 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab0abe1-70ad-4ca3-aa42-26d2d41eb7af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:39.335012 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab0abe1-70ad-4ca3-aa42-26d2d41eb7af service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:39.347448 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1404/2791] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:02:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:02:39.725523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c5eae-db86-4434-9373-ca49aad2ea7b None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:39.726631 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-c47c5eae-db86-4434-9373-ca49aad2ea7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:39.755302 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c5eae-db86-4434-9373-ca49aad2ea7b None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:39.768002 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e6ca6-6632-4de1-8cc3-bc24025ce693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:39.771143 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e6ca6-6632-4de1-8cc3-bc24025ce693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:39.771143 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e6ca6-6632-4de1-8cc3-bc24025ce693 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:39.771143 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e6ca6-6632-4de1-8cc3-bc24025ce693 admin admin] PATH_INFO: `/v3/projects/3f752219ef5443a08876f6dad7c024bd` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:39.772639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879e6ca6-6632-4de1-8cc3-bc24025ce693 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f752219ef5443a08876f6dad7c024bd)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:39.775356 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879e6ca6-6632-4de1-8cc3-bc24025ce693 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:39.819185 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:02:39.840053 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-879e6ca6-6632-4de1-8cc3-bc24025ce693 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f752219ef5443a08876f6dad7c024bd', 'request_id': 'req-879e6ca6-6632-4de1-8cc3-bc24025ce693'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:39.851796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-879e6ca6-6632-4de1-8cc3-bc24025ce693 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f752219ef5443a08876f6dad7c024bd', 'request_id': 'req-879e6ca6-6632-4de1-8cc3-bc24025ce693'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:39.859728 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-879e6ca6-6632-4de1-8cc3-bc24025ce693 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f752219ef5443a08876f6dad7c024bd', 'request_id': 'req-879e6ca6-6632-4de1-8cc3-bc24025ce693'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:39.877649 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-879e6ca6-6632-4de1-8cc3-bc24025ce693 admin admin] The token cache is being invalidate because project 3f752219ef5443a08876f6dad7c024bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:02:39.877928 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-879e6ca6-6632-4de1-8cc3-bc24025ce693 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:39.878995 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-879e6ca6-6632-4de1-8cc3-bc24025ce693 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:39.882381 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1388/2792] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:02:39 2022] DELETE /identity/v3/projects/3f752219ef5443a08876f6dad7c024bd => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:02:39.886759 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab0abe1-70ad-4ca3-aa42-26d2d41eb7af service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:39.899891 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab0abe1-70ad-4ca3-aa42-26d2d41eb7af service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:39.988270 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025270b5-22f7-42f2-aceb-e827247704f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:39.988843 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025270b5-22f7-42f2-aceb-e827247704f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:39.989275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025270b5-22f7-42f2-aceb-e827247704f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:39.989617 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025270b5-22f7-42f2-aceb-e827247704f2 admin admin] PATH_INFO: `/v3/users/116e208b0856494fa10ea42276b2b9cb` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:39.990855 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025270b5-22f7-42f2-aceb-e827247704f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=116e208b0856494fa10ea42276b2b9cb)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:39.992869 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025270b5-22f7-42f2-aceb-e827247704f2 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:40.071623 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-025270b5-22f7-42f2-aceb-e827247704f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '116e208b0856494fa10ea42276b2b9cb'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:40.087860 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-025270b5-22f7-42f2-aceb-e827247704f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '116e208b0856494fa10ea42276b2b9cb'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:40.098472 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-025270b5-22f7-42f2-aceb-e827247704f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '116e208b0856494fa10ea42276b2b9cb'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:40.109511 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-025270b5-22f7-42f2-aceb-e827247704f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '116e208b0856494fa10ea42276b2b9cb'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:40.110698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-025270b5-22f7-42f2-aceb-e827247704f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '116e208b0856494fa10ea42276b2b9cb'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:40.118158 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1405/2793] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:02:39 2022] DELETE /identity/v3/users/116e208b0856494fa10ea42276b2b9cb => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:02:40.257373 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e6ca6-6632-4de1-8cc3-bc24025ce693 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:40.260639 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-879e6ca6-6632-4de1-8cc3-bc24025ce693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:40.431094 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e6ca6-6632-4de1-8cc3-bc24025ce693 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:40.629802 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef3f6d-9366-4c74-9386-4ccee94eacc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:40.631539 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef3f6d-9366-4c74-9386-4ccee94eacc2 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:40.632096 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef3f6d-9366-4c74-9386-4ccee94eacc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:40.632660 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef3f6d-9366-4c74-9386-4ccee94eacc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:40.639253 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025270b5-22f7-42f2-aceb-e827247704f2 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:40.641551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-025270b5-22f7-42f2-aceb-e827247704f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:40.641755 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-6fef3f6d-9366-4c74-9386-4ccee94eacc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:40.662425 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef3f6d-9366-4c74-9386-4ccee94eacc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:40.667944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef3f6d-9366-4c74-9386-4ccee94eacc2 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:40.687824 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1389/2794] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:02:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:02:40.793400 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:02:40.834711 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025270b5-22f7-42f2-aceb-e827247704f2 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:40.853559 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84858d30-eee4-4251-ad99-a322613939c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:40.854840 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84858d30-eee4-4251-ad99-a322613939c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:40.864370 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84858d30-eee4-4251-ad99-a322613939c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:40.865061 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84858d30-eee4-4251-ad99-a322613939c4 admin admin] PATH_INFO: `/v3/projects/ea1952483be74f8e87880f5e3ce7369d` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:40.871727 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84858d30-eee4-4251-ad99-a322613939c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea1952483be74f8e87880f5e3ce7369d)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:40.875525 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84858d30-eee4-4251-ad99-a322613939c4 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:40.965213 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-84858d30-eee4-4251-ad99-a322613939c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea1952483be74f8e87880f5e3ce7369d', 'request_id': 'req-84858d30-eee4-4251-ad99-a322613939c4'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:40.981966 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-84858d30-eee4-4251-ad99-a322613939c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea1952483be74f8e87880f5e3ce7369d', 'request_id': 'req-84858d30-eee4-4251-ad99-a322613939c4'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:40.999049 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-84858d30-eee4-4251-ad99-a322613939c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea1952483be74f8e87880f5e3ce7369d', 'request_id': 'req-84858d30-eee4-4251-ad99-a322613939c4'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:41.030387 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-84858d30-eee4-4251-ad99-a322613939c4 admin admin] The token cache is being invalidate because project ea1952483be74f8e87880f5e3ce7369d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:02:41.030672 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-84858d30-eee4-4251-ad99-a322613939c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:41.031513 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-84858d30-eee4-4251-ad99-a322613939c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:41.033618 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1406/2795] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:02:40 2022] DELETE /identity/v3/projects/ea1952483be74f8e87880f5e3ce7369d => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:02:41.039804 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef3f6d-9366-4c74-9386-4ccee94eacc2 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:41.060260 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef3f6d-9366-4c74-9386-4ccee94eacc2 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:41.193422 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153865f-1f3f-4293-a638-6c38bf705d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:41.194248 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153865f-1f3f-4293-a638-6c38bf705d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:41.194524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153865f-1f3f-4293-a638-6c38bf705d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:41.195063 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9153865f-1f3f-4293-a638-6c38bf705d65 admin admin] PATH_INFO: `/v3/users/14ebf7c1fbf14eeb9456826ff324ae1e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:41.197901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9153865f-1f3f-4293-a638-6c38bf705d65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14ebf7c1fbf14eeb9456826ff324ae1e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:41.201007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9153865f-1f3f-4293-a638-6c38bf705d65 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:41.248969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:02:41.292476 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-9153865f-1f3f-4293-a638-6c38bf705d65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14ebf7c1fbf14eeb9456826ff324ae1e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:41.304422 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-9153865f-1f3f-4293-a638-6c38bf705d65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14ebf7c1fbf14eeb9456826ff324ae1e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:41.317161 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-9153865f-1f3f-4293-a638-6c38bf705d65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14ebf7c1fbf14eeb9456826ff324ae1e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:41.319277 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-9153865f-1f3f-4293-a638-6c38bf705d65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14ebf7c1fbf14eeb9456826ff324ae1e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:41.320883 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-9153865f-1f3f-4293-a638-6c38bf705d65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14ebf7c1fbf14eeb9456826ff324ae1e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:41.375455 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1390/2796] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:02:41 2022] DELETE /identity/v3/users/14ebf7c1fbf14eeb9456826ff324ae1e => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:02:42.005636 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84858d30-eee4-4251-ad99-a322613939c4 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:42.009964 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-84858d30-eee4-4251-ad99-a322613939c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:42.034248 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84858d30-eee4-4251-ad99-a322613939c4 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:42.121579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19661149-f107-4d00-a6c6-9052a5e1d7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:42.123089 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19661149-f107-4d00-a6c6-9052a5e1d7a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:42.123726 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19661149-f107-4d00-a6c6-9052a5e1d7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:42.124142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19661149-f107-4d00-a6c6-9052a5e1d7a6 admin admin] PATH_INFO: `/v3/projects/68fcb05620e04880872c60a67d230ffe` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:42.126004 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19661149-f107-4d00-a6c6-9052a5e1d7a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68fcb05620e04880872c60a67d230ffe)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:42.128367 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19661149-f107-4d00-a6c6-9052a5e1d7a6 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:42.222344 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-19661149-f107-4d00-a6c6-9052a5e1d7a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68fcb05620e04880872c60a67d230ffe', 'request_id': 'req-19661149-f107-4d00-a6c6-9052a5e1d7a6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:42.231271 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-19661149-f107-4d00-a6c6-9052a5e1d7a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68fcb05620e04880872c60a67d230ffe', 'request_id': 'req-19661149-f107-4d00-a6c6-9052a5e1d7a6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:42.245253 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-19661149-f107-4d00-a6c6-9052a5e1d7a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68fcb05620e04880872c60a67d230ffe', 'request_id': 'req-19661149-f107-4d00-a6c6-9052a5e1d7a6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:42.265524 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-19661149-f107-4d00-a6c6-9052a5e1d7a6 admin admin] The token cache is being invalidate because project 68fcb05620e04880872c60a67d230ffe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:02:42.265990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-19661149-f107-4d00-a6c6-9052a5e1d7a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:42.267091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-19661149-f107-4d00-a6c6-9052a5e1d7a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:42.273210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1407/2797] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:02:42 2022] DELETE /identity/v3/projects/68fcb05620e04880872c60a67d230ffe => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:02:42.623528 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153865f-1f3f-4293-a638-6c38bf705d65 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:42.631773 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9153865f-1f3f-4293-a638-6c38bf705d65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:42.750724 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9153865f-1f3f-4293-a638-6c38bf705d65 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:42.830120 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b8e4b-0556-484b-aec3-f14a6fcb8d10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:42.830721 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b8e4b-0556-484b-aec3-f14a6fcb8d10 service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:42.831057 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b8e4b-0556-484b-aec3-f14a6fcb8d10 service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:42.831338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b8e4b-0556-484b-aec3-f14a6fcb8d10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:42.833403 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d36b8e4b-0556-484b-aec3-f14a6fcb8d10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:42.847186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b8e4b-0556-484b-aec3-f14a6fcb8d10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:42.848871 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b8e4b-0556-484b-aec3-f14a6fcb8d10 service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:42.861436 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1391/2798] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:02:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:02:49.315365 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19661149-f107-4d00-a6c6-9052a5e1d7a6 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:49.317298 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-19661149-f107-4d00-a6c6-9052a5e1d7a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:49.334014 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:02:49.338611 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19661149-f107-4d00-a6c6-9052a5e1d7a6 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:49.353607 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ece88-17ce-49ae-919c-9aba86177187 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:49.354495 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ece88-17ce-49ae-919c-9aba86177187 service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:49.355205 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ece88-17ce-49ae-919c-9aba86177187 service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:49.355586 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ece88-17ce-49ae-919c-9aba86177187 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:49.357751 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d02ece88-17ce-49ae-919c-9aba86177187 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:49.380646 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ece88-17ce-49ae-919c-9aba86177187 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:49.382410 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ece88-17ce-49ae-919c-9aba86177187 service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:49.397412 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1408/2799] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:02:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:02:52.509374 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b8e4b-0556-484b-aec3-f14a6fcb8d10 service nova] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:52.512227 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d36b8e4b-0556-484b-aec3-f14a6fcb8d10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:52.535368 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:02:52.585021 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b8e4b-0556-484b-aec3-f14a6fcb8d10 service nova] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:52.590360 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061d8e5-b84b-461c-9df6-e5f208896f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:52.591135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061d8e5-b84b-461c-9df6-e5f208896f0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:52.591320 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061d8e5-b84b-461c-9df6-e5f208896f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:52.591584 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061d8e5-b84b-461c-9df6-e5f208896f0b admin admin] PATH_INFO: `/v3/users/8340ffdcb82f466e99897d99b0908e91` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:52.592815 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f061d8e5-b84b-461c-9df6-e5f208896f0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8340ffdcb82f466e99897d99b0908e91)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:52.594785 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f061d8e5-b84b-461c-9df6-e5f208896f0b admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:52.649037 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-f061d8e5-b84b-461c-9df6-e5f208896f0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8340ffdcb82f466e99897d99b0908e91'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:52.664496 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-f061d8e5-b84b-461c-9df6-e5f208896f0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8340ffdcb82f466e99897d99b0908e91'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:52.677550 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-f061d8e5-b84b-461c-9df6-e5f208896f0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8340ffdcb82f466e99897d99b0908e91'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:52.678199 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-f061d8e5-b84b-461c-9df6-e5f208896f0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8340ffdcb82f466e99897d99b0908e91'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:52.678811 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-f061d8e5-b84b-461c-9df6-e5f208896f0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8340ffdcb82f466e99897d99b0908e91'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:52.698016 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1392/2800] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:02:52 2022] DELETE /identity/v3/users/8340ffdcb82f466e99897d99b0908e91 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:02:52.703193 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdef65-afb7-4a28-9862-289fc539f356 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:52.703705 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdef65-afb7-4a28-9862-289fc539f356 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:52.703853 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdef65-afb7-4a28-9862-289fc539f356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:52.724814 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-8ebdef65-afb7-4a28-9862-289fc539f356 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:02:52.726075 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8ebdef65-afb7-4a28-9862-289fc539f356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:52.793377 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1409/2801] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:02:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:02:52.805058 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061d8e5-b84b-461c-9df6-e5f208896f0b admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:52.806768 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f061d8e5-b84b-461c-9df6-e5f208896f0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:52.873584 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061d8e5-b84b-461c-9df6-e5f208896f0b admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:52.878597 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038951c9-0e83-4277-a0fa-7b727c74f8d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:52.879288 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038951c9-0e83-4277-a0fa-7b727c74f8d3 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:52.879616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038951c9-0e83-4277-a0fa-7b727c74f8d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:52.880010 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038951c9-0e83-4277-a0fa-7b727c74f8d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:52.892071 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038951c9-0e83-4277-a0fa-7b727c74f8d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:52.893802 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038951c9-0e83-4277-a0fa-7b727c74f8d3 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:52.906392 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1393/2802] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:02:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:02:53.249422 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdef65-afb7-4a28-9862-289fc539f356 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:53.251877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-8ebdef65-afb7-4a28-9862-289fc539f356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:53.266000 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdef65-afb7-4a28-9862-289fc539f356 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:53.271942 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b61c6-9526-42a8-8230-8336168e977a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:53.272764 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b61c6-9526-42a8-8230-8336168e977a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:53.273040 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b61c6-9526-42a8-8230-8336168e977a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:53.273426 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b61c6-9526-42a8-8230-8336168e977a admin admin] PATH_INFO: `/v3/projects/fc73b5e9132b482798c0c359a4a5d9be` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:53.275038 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44b61c6-9526-42a8-8230-8336168e977a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc73b5e9132b482798c0c359a4a5d9be)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:53.277498 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44b61c6-9526-42a8-8230-8336168e977a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:53.343480 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c44b61c6-9526-42a8-8230-8336168e977a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc73b5e9132b482798c0c359a4a5d9be', 'request_id': 'req-c44b61c6-9526-42a8-8230-8336168e977a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:53.349677 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c44b61c6-9526-42a8-8230-8336168e977a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc73b5e9132b482798c0c359a4a5d9be', 'request_id': 'req-c44b61c6-9526-42a8-8230-8336168e977a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:53.359591 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c44b61c6-9526-42a8-8230-8336168e977a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc73b5e9132b482798c0c359a4a5d9be', 'request_id': 'req-c44b61c6-9526-42a8-8230-8336168e977a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:53.375511 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c44b61c6-9526-42a8-8230-8336168e977a admin admin] The token cache is being invalidate because project fc73b5e9132b482798c0c359a4a5d9be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:02:53.375815 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c44b61c6-9526-42a8-8230-8336168e977a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:53.376481 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c44b61c6-9526-42a8-8230-8336168e977a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:53.378048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1410/2803] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:02:53 2022] DELETE /identity/v3/projects/fc73b5e9132b482798c0c359a4a5d9be => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:02:53.382370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038951c9-0e83-4277-a0fa-7b727c74f8d3 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:53.393633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038951c9-0e83-4277-a0fa-7b727c74f8d3 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:53.439595 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:02:53.459283 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f3958-ad9f-4c01-a8b5-bf036ebd8ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:53.459925 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f3958-ad9f-4c01-a8b5-bf036ebd8ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:53.460205 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f3958-ad9f-4c01-a8b5-bf036ebd8ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:53.460478 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f3958-ad9f-4c01-a8b5-bf036ebd8ee3 admin admin] PATH_INFO: `/v3/users/1785bb2d8dd54b0eb0d8c4b0312a141b` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:53.461645 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f3958-ad9f-4c01-a8b5-bf036ebd8ee3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1785bb2d8dd54b0eb0d8c4b0312a141b)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:53.463646 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f3958-ad9f-4c01-a8b5-bf036ebd8ee3 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:53.524340 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-476f3958-ad9f-4c01-a8b5-bf036ebd8ee3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1785bb2d8dd54b0eb0d8c4b0312a141b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:53.542755 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-476f3958-ad9f-4c01-a8b5-bf036ebd8ee3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1785bb2d8dd54b0eb0d8c4b0312a141b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:53.551284 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-476f3958-ad9f-4c01-a8b5-bf036ebd8ee3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1785bb2d8dd54b0eb0d8c4b0312a141b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:53.552159 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-476f3958-ad9f-4c01-a8b5-bf036ebd8ee3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1785bb2d8dd54b0eb0d8c4b0312a141b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:53.552962 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-476f3958-ad9f-4c01-a8b5-bf036ebd8ee3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1785bb2d8dd54b0eb0d8c4b0312a141b'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:53.572272 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1394/2804] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:02:53 2022] DELETE /identity/v3/users/1785bb2d8dd54b0eb0d8c4b0312a141b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 09:02:53.878315 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b61c6-9526-42a8-8230-8336168e977a admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:53.881646 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c44b61c6-9526-42a8-8230-8336168e977a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:53.900473 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b61c6-9526-42a8-8230-8336168e977a admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:53.965919 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29bdee-e9a4-44f0-9587-49cfa361aecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:53.966597 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29bdee-e9a4-44f0-9587-49cfa361aecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:53.966878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29bdee-e9a4-44f0-9587-49cfa361aecc admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:53.967184 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29bdee-e9a4-44f0-9587-49cfa361aecc admin admin] PATH_INFO: `/v3/projects/dcaaf344e67c46829646fcc09361aed8` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:53.968364 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29bdee-e9a4-44f0-9587-49cfa361aecc admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcaaf344e67c46829646fcc09361aed8)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:53.970367 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29bdee-e9a4-44f0-9587-49cfa361aecc admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:53.989325 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:02:54.028603 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2f29bdee-e9a4-44f0-9587-49cfa361aecc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcaaf344e67c46829646fcc09361aed8', 'request_id': 'req-2f29bdee-e9a4-44f0-9587-49cfa361aecc'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:54.035715 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2f29bdee-e9a4-44f0-9587-49cfa361aecc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcaaf344e67c46829646fcc09361aed8', 'request_id': 'req-2f29bdee-e9a4-44f0-9587-49cfa361aecc'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:54.045650 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2f29bdee-e9a4-44f0-9587-49cfa361aecc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcaaf344e67c46829646fcc09361aed8', 'request_id': 'req-2f29bdee-e9a4-44f0-9587-49cfa361aecc'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:54.058572 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2f29bdee-e9a4-44f0-9587-49cfa361aecc admin admin] The token cache is being invalidate because project dcaaf344e67c46829646fcc09361aed8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:02:54.058850 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2f29bdee-e9a4-44f0-9587-49cfa361aecc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:54.059539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2f29bdee-e9a4-44f0-9587-49cfa361aecc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:54.061033 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1411/2805] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:02:53 2022] DELETE /identity/v3/projects/dcaaf344e67c46829646fcc09361aed8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:02:54.066665 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17bbd5-8790-430d-88df-cc3573ae7007 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:54.066980 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17bbd5-8790-430d-88df-cc3573ae7007 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:54.067252 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17bbd5-8790-430d-88df-cc3573ae7007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:54.090340 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-9c17bbd5-8790-430d-88df-cc3573ae7007 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:02:54.091584 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9c17bbd5-8790-430d-88df-cc3573ae7007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:54.161108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1395/2806] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:02:54.171726 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3738ae5-4607-4bb9-97d3-3952c76cec76 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:54.172048 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3738ae5-4607-4bb9-97d3-3952c76cec76 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:54.172261 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3738ae5-4607-4bb9-97d3-3952c76cec76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:54.194345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-f3738ae5-4607-4bb9-97d3-3952c76cec76 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:02:54.195716 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f3738ae5-4607-4bb9-97d3-3952c76cec76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:54.205011 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1412/2807] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:02:54.209892 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17bbd5-8790-430d-88df-cc3573ae7007 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:54.220036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17bbd5-8790-430d-88df-cc3573ae7007 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:54.225227 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad0986-650b-4cd4-9616-be6abee71961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:54.226033 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad0986-650b-4cd4-9616-be6abee71961 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:54.226418 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad0986-650b-4cd4-9616-be6abee71961 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:54.226795 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad0986-650b-4cd4-9616-be6abee71961 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:54.227669 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad0986-650b-4cd4-9616-be6abee71961 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 09:02:54.228067 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad0986-650b-4cd4-9616-be6abee71961 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:54.230107 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad0986-650b-4cd4-9616-be6abee71961 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:54.243358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1396/2808] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 09:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 09:02:54.248087 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3738ae5-4607-4bb9-97d3-3952c76cec76 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:54.259033 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3738ae5-4607-4bb9-97d3-3952c76cec76 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:54.264367 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca61ecd-a33a-41c5-a4dc-09608b22984b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:54.264986 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca61ecd-a33a-41c5-a4dc-09608b22984b admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:54.265266 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca61ecd-a33a-41c5-a4dc-09608b22984b admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:54.265539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca61ecd-a33a-41c5-a4dc-09608b22984b admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:54.266175 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca61ecd-a33a-41c5-a4dc-09608b22984b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:54.268146 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca61ecd-a33a-41c5-a4dc-09608b22984b admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:54.305446 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1413/2809] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:02:54 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:02:54.310061 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad0986-650b-4cd4-9616-be6abee71961 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:54.321762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad0986-650b-4cd4-9616-be6abee71961 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:54.372235 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:02:54.391254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a2016-07f4-4d33-8066-3cdaf88d6624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:54.391926 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a2016-07f4-4d33-8066-3cdaf88d6624 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:54.392467 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a2016-07f4-4d33-8066-3cdaf88d6624 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:54.392467 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a2016-07f4-4d33-8066-3cdaf88d6624 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:54.393507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5a2016-07f4-4d33-8066-3cdaf88d6624 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:54.395036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5a2016-07f4-4d33-8066-3cdaf88d6624 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:54.419464 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1397/2810] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:02:54 2022] POST /identity/v3/users => generated 283 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 09:02:54.424455 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca61ecd-a33a-41c5-a4dc-09608b22984b admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:54.435845 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca61ecd-a33a-41c5-a4dc-09608b22984b admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:54.440592 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4121fca-0d1b-4eb4-a0f6-e0b6f9751c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:54.441211 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4121fca-0d1b-4eb4-a0f6-e0b6f9751c8d admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:54.441483 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4121fca-0d1b-4eb4-a0f6-e0b6f9751c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:54.441747 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4121fca-0d1b-4eb4-a0f6-e0b6f9751c8d admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:54.442522 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4121fca-0d1b-4eb4-a0f6-e0b6f9751c8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:54.444474 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4121fca-0d1b-4eb4-a0f6-e0b6f9751c8d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:54.455690 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1414/2811] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:02:54.460347 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a2016-07f4-4d33-8066-3cdaf88d6624 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:54.471737 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a2016-07f4-4d33-8066-3cdaf88d6624 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:54.477112 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807fad48-d651-4452-a53d-90d8ce45fc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:54.477927 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807fad48-d651-4452-a53d-90d8ce45fc39 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:54.478199 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807fad48-d651-4452-a53d-90d8ce45fc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:54.478460 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807fad48-d651-4452-a53d-90d8ce45fc39 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:54.479143 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807fad48-d651-4452-a53d-90d8ce45fc39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:54.480752 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807fad48-d651-4452-a53d-90d8ce45fc39 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:54.492671 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1398/2812] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:02:54.498445 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4121fca-0d1b-4eb4-a0f6-e0b6f9751c8d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:54.511007 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4121fca-0d1b-4eb4-a0f6-e0b6f9751c8d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:54.517501 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f385864-2236-4786-9593-8d65b3fc92af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:54.518195 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f385864-2236-4786-9593-8d65b3fc92af admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:54.518502 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f385864-2236-4786-9593-8d65b3fc92af admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:54.518805 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f385864-2236-4786-9593-8d65b3fc92af admin admin] PATH_INFO: `/v3/projects/e4b74b346c534715b6a91eeadc6421f5/users/81d045953b03493ca004607fecfcd918/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:54.537162 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f385864-2236-4786-9593-8d65b3fc92af admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4b74b346c534715b6a91eeadc6421f5, user_id=81d045953b03493ca004607fecfcd918, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:54.540012 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f385864-2236-4786-9593-8d65b3fc92af admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:54.556732 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1415/2813] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:02:54 2022] PUT /identity/v3/projects/e4b74b346c534715b6a91eeadc6421f5/users/81d045953b03493ca004607fecfcd918/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:02:54.567723 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82f682-7c60-4366-b689-1e1ebcc31d01 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:54.568339 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82f682-7c60-4366-b689-1e1ebcc31d01 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:54.568339 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82f682-7c60-4366-b689-1e1ebcc31d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:54.601959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-df82f682-7c60-4366-b689-1e1ebcc31d01 None None] MFA Rules not processed for user `81d045953b03493ca004607fecfcd918`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:02:54.603219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-df82f682-7c60-4366-b689-1e1ebcc31d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:54.673764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1399/2814] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:02:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:02:54.678799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f385864-2236-4786-9593-8d65b3fc92af admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:54.748508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f385864-2236-4786-9593-8d65b3fc92af admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:54.756414 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b151de-fe6a-4efc-8ae4-4a122fb4fc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:54.757203 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b151de-fe6a-4efc-8ae4-4a122fb4fc1e admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:54.757643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b151de-fe6a-4efc-8ae4-4a122fb4fc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:54.757913 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b151de-fe6a-4efc-8ae4-4a122fb4fc1e admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:54.758958 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b151de-fe6a-4efc-8ae4-4a122fb4fc1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:54.761012 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b151de-fe6a-4efc-8ae4-4a122fb4fc1e admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:54.804608 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1416/2815] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:02:54 2022] POST /identity/v3/projects => generated 365 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:02:54.811926 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df82f682-7c60-4366-b689-1e1ebcc31d01 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:54.831507 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df82f682-7c60-4366-b689-1e1ebcc31d01 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:54.900672 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0af886-38dd-44f5-909c-d9aa473bd882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:54.901298 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0af886-38dd-44f5-909c-d9aa473bd882 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:54.901575 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0af886-38dd-44f5-909c-d9aa473bd882 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:54.901844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0af886-38dd-44f5-909c-d9aa473bd882 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:54.902487 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0af886-38dd-44f5-909c-d9aa473bd882 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:54.904568 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0af886-38dd-44f5-909c-d9aa473bd882 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:54.936250 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1400/2816] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:02:54 2022] POST /identity/v3/users => generated 289 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 09:02:54.940516 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b151de-fe6a-4efc-8ae4-4a122fb4fc1e admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:54.951949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b151de-fe6a-4efc-8ae4-4a122fb4fc1e admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:54.962133 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395a3de-3166-4a93-b8e6-299f8b1f3e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:54.962907 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395a3de-3166-4a93-b8e6-299f8b1f3e12 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:54.963323 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395a3de-3166-4a93-b8e6-299f8b1f3e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:54.963700 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395a3de-3166-4a93-b8e6-299f8b1f3e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:54.964612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395a3de-3166-4a93-b8e6-299f8b1f3e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:54.966437 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395a3de-3166-4a93-b8e6-299f8b1f3e12 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:54.979067 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1417/2817] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:02:54.983746 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0af886-38dd-44f5-909c-d9aa473bd882 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:54.994292 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0af886-38dd-44f5-909c-d9aa473bd882 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:54.999135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b975b-0c4c-4fe7-9a4b-f9a90ba8d2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:54.999791 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b975b-0c4c-4fe7-9a4b-f9a90ba8d2e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:55.000088 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b975b-0c4c-4fe7-9a4b-f9a90ba8d2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:55.000358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b975b-0c4c-4fe7-9a4b-f9a90ba8d2e9 admin admin] PATH_INFO: `/v3/projects/8124db24c02b49bd9b89fc8f1f38d2c1/users/7811bca9d6ba4fc4848edb054ad056c5/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:55.018039 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b975b-0c4c-4fe7-9a4b-f9a90ba8d2e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8124db24c02b49bd9b89fc8f1f38d2c1, user_id=7811bca9d6ba4fc4848edb054ad056c5, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:55.020813 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b975b-0c4c-4fe7-9a4b-f9a90ba8d2e9 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:55.036694 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1401/2818] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:02:54 2022] PUT /identity/v3/projects/8124db24c02b49bd9b89fc8f1f38d2c1/users/7811bca9d6ba4fc4848edb054ad056c5/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:02:55.040551 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395a3de-3166-4a93-b8e6-299f8b1f3e12 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:55.051315 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395a3de-3166-4a93-b8e6-299f8b1f3e12 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:55.065512 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:02:55.118122 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a15cf6-d55c-4fbc-86e2-acd7e1c5d2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:55.118408 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a15cf6-d55c-4fbc-86e2-acd7e1c5d2c1 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:55.118520 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a15cf6-d55c-4fbc-86e2-acd7e1c5d2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:55.118801 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a15cf6-d55c-4fbc-86e2-acd7e1c5d2c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:55.119582 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a15cf6-d55c-4fbc-86e2-acd7e1c5d2c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:55.121058 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a15cf6-d55c-4fbc-86e2-acd7e1c5d2c1 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:55.132077 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1418/2819] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:02:55.137714 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b975b-0c4c-4fe7-9a4b-f9a90ba8d2e9 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:55.149577 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b975b-0c4c-4fe7-9a4b-f9a90ba8d2e9 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:55.155338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595b9ba-8a2f-4c39-a596-23712ec533e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:55.156010 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595b9ba-8a2f-4c39-a596-23712ec533e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:55.156297 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595b9ba-8a2f-4c39-a596-23712ec533e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:55.156586 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595b9ba-8a2f-4c39-a596-23712ec533e0 admin admin] PATH_INFO: `/v3/projects/8124db24c02b49bd9b89fc8f1f38d2c1/users/7811bca9d6ba4fc4848edb054ad056c5/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:55.158782 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595b9ba-8a2f-4c39-a596-23712ec533e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8124db24c02b49bd9b89fc8f1f38d2c1, user_id=7811bca9d6ba4fc4848edb054ad056c5, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:55.161844 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595b9ba-8a2f-4c39-a596-23712ec533e0 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:55.174809 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1402/2820] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:02:55 2022] PUT /identity/v3/projects/8124db24c02b49bd9b89fc8f1f38d2c1/users/7811bca9d6ba4fc4848edb054ad056c5/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:02:55.247557 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8d055-c56b-4818-a3fd-dcf8765d3cb4 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:55.247839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8d055-c56b-4818-a3fd-dcf8765d3cb4 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:55.248099 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8d055-c56b-4818-a3fd-dcf8765d3cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:55.280749 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-d6f8d055-c56b-4818-a3fd-dcf8765d3cb4 None None] MFA Rules not processed for user `7811bca9d6ba4fc4848edb054ad056c5`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:02:55.282003 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-d6f8d055-c56b-4818-a3fd-dcf8765d3cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:55.380529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1419/2821] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:02:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 09:02:55.394352 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595b9ba-8a2f-4c39-a596-23712ec533e0 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:55.396200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a595b9ba-8a2f-4c39-a596-23712ec533e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:55.408064 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:02:55.467312 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595b9ba-8a2f-4c39-a596-23712ec533e0 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:55.471456 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf3912-a9e6-4208-8dfe-bc52713949ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:55.472195 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf3912-a9e6-4208-8dfe-bc52713949ae service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:55.472469 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf3912-a9e6-4208-8dfe-bc52713949ae service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:55.472732 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf3912-a9e6-4208-8dfe-bc52713949ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:55.483945 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf3912-a9e6-4208-8dfe-bc52713949ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:55.485998 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf3912-a9e6-4208-8dfe-bc52713949ae service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:55.497940 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1403/2822] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:02:55 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:02:55.588014 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8d055-c56b-4818-a3fd-dcf8765d3cb4 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:55.600556 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8d055-c56b-4818-a3fd-dcf8765d3cb4 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:55.604963 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba294a-030b-4ecf-98b6-f2bbe02d098c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:55.605749 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba294a-030b-4ecf-98b6-f2bbe02d098c service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:55.606110 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba294a-030b-4ecf-98b6-f2bbe02d098c service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:55.606425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba294a-030b-4ecf-98b6-f2bbe02d098c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:55.693725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba294a-030b-4ecf-98b6-f2bbe02d098c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:55.695523 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba294a-030b-4ecf-98b6-f2bbe02d098c service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:55.709275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1420/2823] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:02:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 09:02:58.556506 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf3912-a9e6-4208-8dfe-bc52713949ae service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:58.640200 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf3912-a9e6-4208-8dfe-bc52713949ae service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:58.645846 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91fbb63-aa82-49cd-8247-f33a6f591701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:58.646703 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fbb63-aa82-49cd-8247-f33a6f591701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:58.647012 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fbb63-aa82-49cd-8247-f33a6f591701 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:58.647340 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fbb63-aa82-49cd-8247-f33a6f591701 admin admin] PATH_INFO: `/v3/users/81d045953b03493ca004607fecfcd918` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:58.648598 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91fbb63-aa82-49cd-8247-f33a6f591701 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81d045953b03493ca004607fecfcd918)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:58.650589 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91fbb63-aa82-49cd-8247-f33a6f591701 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:58.711715 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e91fbb63-aa82-49cd-8247-f33a6f591701 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81d045953b03493ca004607fecfcd918'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:58.721643 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e91fbb63-aa82-49cd-8247-f33a6f591701 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81d045953b03493ca004607fecfcd918'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:58.732248 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e91fbb63-aa82-49cd-8247-f33a6f591701 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81d045953b03493ca004607fecfcd918'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:58.733064 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e91fbb63-aa82-49cd-8247-f33a6f591701 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81d045953b03493ca004607fecfcd918'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:58.733694 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-e91fbb63-aa82-49cd-8247-f33a6f591701 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81d045953b03493ca004607fecfcd918'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:58.760747 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1404/2824] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:02:58 2022] DELETE /identity/v3/users/81d045953b03493ca004607fecfcd918 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 09:02:58.766766 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a3872-dbc5-4648-b573-a197e0a63768 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:58.767342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a3872-dbc5-4648-b573-a197e0a63768 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:58.767453 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a3872-dbc5-4648-b573-a197e0a63768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:58.789316 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ae0a3872-dbc5-4648-b573-a197e0a63768 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:02:58.790604 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ae0a3872-dbc5-4648-b573-a197e0a63768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:58.834092 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=100 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:02:58.873990 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1421/2825] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:02:58.889992 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91fbb63-aa82-49cd-8247-f33a6f591701 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:58.893658 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e91fbb63-aa82-49cd-8247-f33a6f591701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:58.969004 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:02:59.014567 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91fbb63-aa82-49cd-8247-f33a6f591701 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:59.018745 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0958797-64f2-4f96-9067-6b690b8a0204 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:59.019463 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0958797-64f2-4f96-9067-6b690b8a0204 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:59.019698 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0958797-64f2-4f96-9067-6b690b8a0204 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:59.019975 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0958797-64f2-4f96-9067-6b690b8a0204 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:59.033398 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0958797-64f2-4f96-9067-6b690b8a0204 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:59.034928 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0958797-64f2-4f96-9067-6b690b8a0204 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:59.046735 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1405/2826] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:02:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:02:59.387320 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a3872-dbc5-4648-b573-a197e0a63768 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:59.389958 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ae0a3872-dbc5-4648-b573-a197e0a63768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:02:59.406251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a3872-dbc5-4648-b573-a197e0a63768 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:59.411510 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fecf36-9f4d-423f-b89f-500dc8aa693a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:59.412188 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fecf36-9f4d-423f-b89f-500dc8aa693a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:59.412469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fecf36-9f4d-423f-b89f-500dc8aa693a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:59.412740 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fecf36-9f4d-423f-b89f-500dc8aa693a admin admin] PATH_INFO: `/v3/projects/e4b74b346c534715b6a91eeadc6421f5` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:59.415215 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fecf36-9f4d-423f-b89f-500dc8aa693a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4b74b346c534715b6a91eeadc6421f5)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:59.417183 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fecf36-9f4d-423f-b89f-500dc8aa693a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:59.481080 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c7fecf36-9f4d-423f-b89f-500dc8aa693a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4b74b346c534715b6a91eeadc6421f5', 'request_id': 'req-c7fecf36-9f4d-423f-b89f-500dc8aa693a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:59.488951 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c7fecf36-9f4d-423f-b89f-500dc8aa693a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4b74b346c534715b6a91eeadc6421f5', 'request_id': 'req-c7fecf36-9f4d-423f-b89f-500dc8aa693a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:59.501367 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c7fecf36-9f4d-423f-b89f-500dc8aa693a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4b74b346c534715b6a91eeadc6421f5', 'request_id': 'req-c7fecf36-9f4d-423f-b89f-500dc8aa693a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:59.522309 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c7fecf36-9f4d-423f-b89f-500dc8aa693a admin admin] The token cache is being invalidate because project e4b74b346c534715b6a91eeadc6421f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:02:59.522669 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c7fecf36-9f4d-423f-b89f-500dc8aa693a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:59.523318 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-c7fecf36-9f4d-423f-b89f-500dc8aa693a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:59.524827 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1422/2827] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:02:59 2022] DELETE /identity/v3/projects/e4b74b346c534715b6a91eeadc6421f5 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:02:59.529823 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0958797-64f2-4f96-9067-6b690b8a0204 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:02:59.547559 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0958797-64f2-4f96-9067-6b690b8a0204 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:02:59.615060 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4a36c0-a74c-43b8-acc5-fe38009e2184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:02:59.615767 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a36c0-a74c-43b8-acc5-fe38009e2184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:02:59.615979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a36c0-a74c-43b8-acc5-fe38009e2184 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:02:59.616249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4a36c0-a74c-43b8-acc5-fe38009e2184 admin admin] PATH_INFO: `/v3/users/7811bca9d6ba4fc4848edb054ad056c5` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:02:59.617437 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4a36c0-a74c-43b8-acc5-fe38009e2184 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7811bca9d6ba4fc4848edb054ad056c5)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:02:59.619565 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4a36c0-a74c-43b8-acc5-fe38009e2184 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:02:59.685698 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-dc4a36c0-a74c-43b8-acc5-fe38009e2184 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7811bca9d6ba4fc4848edb054ad056c5'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:59.695970 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-dc4a36c0-a74c-43b8-acc5-fe38009e2184 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7811bca9d6ba4fc4848edb054ad056c5'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:59.706355 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-dc4a36c0-a74c-43b8-acc5-fe38009e2184 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7811bca9d6ba4fc4848edb054ad056c5'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:59.707217 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-dc4a36c0-a74c-43b8-acc5-fe38009e2184 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7811bca9d6ba4fc4848edb054ad056c5'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:59.707944 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-dc4a36c0-a74c-43b8-acc5-fe38009e2184 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7811bca9d6ba4fc4848edb054ad056c5'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:02:59.730446 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1406/2828] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:02:59 2022] DELETE /identity/v3/users/7811bca9d6ba4fc4848edb054ad056c5 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:03:00.073477 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fecf36-9f4d-423f-b89f-500dc8aa693a admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:00.075695 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-c7fecf36-9f4d-423f-b89f-500dc8aa693a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:03:00.095949 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fecf36-9f4d-423f-b89f-500dc8aa693a admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:00.179778 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac317f8-c30d-417c-b448-e3804c12a17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:00.180679 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac317f8-c30d-417c-b448-e3804c12a17c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:00.181069 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac317f8-c30d-417c-b448-e3804c12a17c admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:00.181353 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac317f8-c30d-417c-b448-e3804c12a17c admin admin] PATH_INFO: `/v3/projects/8124db24c02b49bd9b89fc8f1f38d2c1` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:00.182855 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac317f8-c30d-417c-b448-e3804c12a17c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8124db24c02b49bd9b89fc8f1f38d2c1)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:00.185057 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac317f8-c30d-417c-b448-e3804c12a17c admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:00.269897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5ac317f8-c30d-417c-b448-e3804c12a17c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8124db24c02b49bd9b89fc8f1f38d2c1', 'request_id': 'req-5ac317f8-c30d-417c-b448-e3804c12a17c'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:00.277080 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5ac317f8-c30d-417c-b448-e3804c12a17c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8124db24c02b49bd9b89fc8f1f38d2c1', 'request_id': 'req-5ac317f8-c30d-417c-b448-e3804c12a17c'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:00.281212 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:03:00.289237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5ac317f8-c30d-417c-b448-e3804c12a17c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8124db24c02b49bd9b89fc8f1f38d2c1', 'request_id': 'req-5ac317f8-c30d-417c-b448-e3804c12a17c'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:00.304336 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5ac317f8-c30d-417c-b448-e3804c12a17c admin admin] The token cache is being invalidate because project 8124db24c02b49bd9b89fc8f1f38d2c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:03:00.304710 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5ac317f8-c30d-417c-b448-e3804c12a17c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:00.305301 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-5ac317f8-c30d-417c-b448-e3804c12a17c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:00.306830 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1423/2829] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:03:00 2022] DELETE /identity/v3/projects/8124db24c02b49bd9b89fc8f1f38d2c1 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:03:00.313588 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e562b-d070-487c-bacc-6adc449d25e0 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:00.313963 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e562b-d070-487c-bacc-6adc449d25e0 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:00.314302 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697e562b-d070-487c-bacc-6adc449d25e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:00.341949 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-697e562b-d070-487c-bacc-6adc449d25e0 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:03:00.343193 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-697e562b-d070-487c-bacc-6adc449d25e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:03:00.404989 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:03:00.422090 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1407/2830] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:03:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 09:03:00.430309 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da03bf2-0610-48d0-a769-85983957d81f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:00.430585 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da03bf2-0610-48d0-a769-85983957d81f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:00.430862 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da03bf2-0610-48d0-a769-85983957d81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:00.452156 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-3da03bf2-0610-48d0-a769-85983957d81f None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:03:00.453404 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-3da03bf2-0610-48d0-a769-85983957d81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:03:00.462978 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1424/2831] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:03:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:03:00.467774 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e562b-d070-487c-bacc-6adc449d25e0 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:00.489216 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697e562b-d070-487c-bacc-6adc449d25e0 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:00.494080 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba008d8a-b94a-4c22-8bd0-ca3605c7da41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:00.494693 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba008d8a-b94a-4c22-8bd0-ca3605c7da41 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:00.494998 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba008d8a-b94a-4c22-8bd0-ca3605c7da41 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:00.495262 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba008d8a-b94a-4c22-8bd0-ca3605c7da41 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:00.495910 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba008d8a-b94a-4c22-8bd0-ca3605c7da41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 09:03:00.496204 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba008d8a-b94a-4c22-8bd0-ca3605c7da41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:00.497796 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba008d8a-b94a-4c22-8bd0-ca3605c7da41 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:00.511427 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1408/2832] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 09:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 09:03:00.516442 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da03bf2-0610-48d0-a769-85983957d81f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:00.526144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da03bf2-0610-48d0-a769-85983957d81f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:00.530976 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514d7a4-1f62-456e-a8a7-cb590ea3c8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:00.531593 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514d7a4-1f62-456e-a8a7-cb590ea3c8da admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:00.531877 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514d7a4-1f62-456e-a8a7-cb590ea3c8da admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:00.532149 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514d7a4-1f62-456e-a8a7-cb590ea3c8da admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:00.532783 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514d7a4-1f62-456e-a8a7-cb590ea3c8da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:00.534697 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514d7a4-1f62-456e-a8a7-cb590ea3c8da admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:00.564714 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1425/2833] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:03:00 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:03:00.569161 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba008d8a-b94a-4c22-8bd0-ca3605c7da41 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:00.583379 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba008d8a-b94a-4c22-8bd0-ca3605c7da41 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:00.645366 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e05db9-d164-4be7-8361-791fc91ab6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:00.646327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e05db9-d164-4be7-8361-791fc91ab6b7 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:00.646327 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e05db9-d164-4be7-8361-791fc91ab6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:00.646582 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e05db9-d164-4be7-8361-791fc91ab6b7 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:00.647267 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e05db9-d164-4be7-8361-791fc91ab6b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:00.649195 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e05db9-d164-4be7-8361-791fc91ab6b7 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:00.672587 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1409/2834] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:03:00 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 09:03:00.678464 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514d7a4-1f62-456e-a8a7-cb590ea3c8da admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:00.689727 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514d7a4-1f62-456e-a8a7-cb590ea3c8da admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:00.694650 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d49e4-9f04-411a-81d1-3dcd1fddecef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:00.695331 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d49e4-9f04-411a-81d1-3dcd1fddecef admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:00.695607 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d49e4-9f04-411a-81d1-3dcd1fddecef admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:00.695875 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d49e4-9f04-411a-81d1-3dcd1fddecef admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:00.696515 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d49e4-9f04-411a-81d1-3dcd1fddecef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:00.698151 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d49e4-9f04-411a-81d1-3dcd1fddecef admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:00.709918 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1426/2835] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:03:00.714443 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e05db9-d164-4be7-8361-791fc91ab6b7 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:00.725028 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e05db9-d164-4be7-8361-791fc91ab6b7 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:00.730111 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe986249-5266-42a5-92e9-51d1552b0631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:00.730747 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe986249-5266-42a5-92e9-51d1552b0631 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:00.731198 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe986249-5266-42a5-92e9-51d1552b0631 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:00.731491 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe986249-5266-42a5-92e9-51d1552b0631 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:00.732151 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe986249-5266-42a5-92e9-51d1552b0631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:00.733753 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe986249-5266-42a5-92e9-51d1552b0631 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:00.745829 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1410/2836] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 09:03:00.751656 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d49e4-9f04-411a-81d1-3dcd1fddecef admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:00.766066 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d49e4-9f04-411a-81d1-3dcd1fddecef admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:00.773214 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815beaf5-7c54-4614-bc21-c073a56a1f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:00.773879 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815beaf5-7c54-4614-bc21-c073a56a1f3a admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:00.774158 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815beaf5-7c54-4614-bc21-c073a56a1f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:00.774430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815beaf5-7c54-4614-bc21-c073a56a1f3a admin admin] PATH_INFO: `/v3/projects/0975ce62e1d64b39aec547ff4b211926/users/8cb76fa34bd5477c9bedc295efc0e62a/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:00.793319 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815beaf5-7c54-4614-bc21-c073a56a1f3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0975ce62e1d64b39aec547ff4b211926, user_id=8cb76fa34bd5477c9bedc295efc0e62a, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:00.796044 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815beaf5-7c54-4614-bc21-c073a56a1f3a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:00.815102 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1427/2837] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:03:00 2022] PUT /identity/v3/projects/0975ce62e1d64b39aec547ff4b211926/users/8cb76fa34bd5477c9bedc295efc0e62a/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:03:00.828011 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec5814-a402-490d-8ac3-3269d77c6ddb None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:00.828435 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec5814-a402-490d-8ac3-3269d77c6ddb None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:00.828868 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec5814-a402-490d-8ac3-3269d77c6ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:00.862137 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-35ec5814-a402-490d-8ac3-3269d77c6ddb None None] MFA Rules not processed for user `8cb76fa34bd5477c9bedc295efc0e62a`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:03:00.863534 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-35ec5814-a402-490d-8ac3-3269d77c6ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:03:00.944391 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1411/2838] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:03:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:03:00.949117 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815beaf5-7c54-4614-bc21-c073a56a1f3a admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:01.015192 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815beaf5-7c54-4614-bc21-c073a56a1f3a admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:01.020289 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f04fee-d5f2-4022-8d96-e082ceebfb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:01.020910 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f04fee-d5f2-4022-8d96-e082ceebfb03 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:01.021358 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f04fee-d5f2-4022-8d96-e082ceebfb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:01.021638 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f04fee-d5f2-4022-8d96-e082ceebfb03 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:01.022308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f04fee-d5f2-4022-8d96-e082ceebfb03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:01.024493 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f04fee-d5f2-4022-8d96-e082ceebfb03 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:01.056101 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1428/2839] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:03:00 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:03:01.060613 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ec5814-a402-490d-8ac3-3269d77c6ddb None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:01.076448 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ec5814-a402-490d-8ac3-3269d77c6ddb None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:01.141182 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:03:01.158476 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f349a-cfad-4e54-b258-f1d834db2574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:01.159133 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f349a-cfad-4e54-b258-f1d834db2574 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:01.159599 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f349a-cfad-4e54-b258-f1d834db2574 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:01.160067 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f349a-cfad-4e54-b258-f1d834db2574 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:01.161196 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f349a-cfad-4e54-b258-f1d834db2574 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:01.164751 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f349a-cfad-4e54-b258-f1d834db2574 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:01.195390 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1412/2840] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:03:01 2022] POST /identity/v3/users => generated 295 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:03:01.200123 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f04fee-d5f2-4022-8d96-e082ceebfb03 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:01.211434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f04fee-d5f2-4022-8d96-e082ceebfb03 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:01.216471 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4849f2-3c1c-47f9-8363-da8e9b4bbccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:01.217170 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4849f2-3c1c-47f9-8363-da8e9b4bbccb admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:01.217443 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4849f2-3c1c-47f9-8363-da8e9b4bbccb admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:01.217708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4849f2-3c1c-47f9-8363-da8e9b4bbccb admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:01.218358 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4849f2-3c1c-47f9-8363-da8e9b4bbccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:01.220077 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4849f2-3c1c-47f9-8363-da8e9b4bbccb admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:01.233153 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1429/2841] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:03:01.237834 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f349a-cfad-4e54-b258-f1d834db2574 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:01.251567 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f349a-cfad-4e54-b258-f1d834db2574 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:01.258072 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21565d6-aaab-4b28-8d13-358ff6f217e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:01.259038 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21565d6-aaab-4b28-8d13-358ff6f217e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:01.259483 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21565d6-aaab-4b28-8d13-358ff6f217e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:01.260021 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21565d6-aaab-4b28-8d13-358ff6f217e5 admin admin] PATH_INFO: `/v3/projects/bddc5761485d4b28abd0501385ac0711/users/99df32ef8b264a0c8be44562148ae99e/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:01.278979 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21565d6-aaab-4b28-8d13-358ff6f217e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bddc5761485d4b28abd0501385ac0711, user_id=99df32ef8b264a0c8be44562148ae99e, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:01.281606 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21565d6-aaab-4b28-8d13-358ff6f217e5 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:01.299724 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1413/2842] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:03:01 2022] PUT /identity/v3/projects/bddc5761485d4b28abd0501385ac0711/users/99df32ef8b264a0c8be44562148ae99e/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 09:03:01.304797 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4849f2-3c1c-47f9-8363-da8e9b4bbccb admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:01.323924 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4849f2-3c1c-47f9-8363-da8e9b4bbccb admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:01.394159 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6149c2-adf1-4217-9a46-c35b3289b284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:01.394799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6149c2-adf1-4217-9a46-c35b3289b284 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:01.395103 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6149c2-adf1-4217-9a46-c35b3289b284 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:01.395375 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6149c2-adf1-4217-9a46-c35b3289b284 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:01.396024 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6149c2-adf1-4217-9a46-c35b3289b284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:01.398231 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6149c2-adf1-4217-9a46-c35b3289b284 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:01.399913 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:03:01.414708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1430/2843] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:03:01.416282 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21565d6-aaab-4b28-8d13-358ff6f217e5 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:01.427545 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21565d6-aaab-4b28-8d13-358ff6f217e5 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:01.432520 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11395d0-9f07-4d91-8c35-b17061ab1c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:01.433186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11395d0-9f07-4d91-8c35-b17061ab1c58 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:01.433620 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11395d0-9f07-4d91-8c35-b17061ab1c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:01.433878 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11395d0-9f07-4d91-8c35-b17061ab1c58 admin admin] PATH_INFO: `/v3/projects/bddc5761485d4b28abd0501385ac0711/users/99df32ef8b264a0c8be44562148ae99e/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:01.435822 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11395d0-9f07-4d91-8c35-b17061ab1c58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bddc5761485d4b28abd0501385ac0711, user_id=99df32ef8b264a0c8be44562148ae99e, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:01.438458 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11395d0-9f07-4d91-8c35-b17061ab1c58 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:01.450037 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1414/2844] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:03:01 2022] PUT /identity/v3/projects/bddc5761485d4b28abd0501385ac0711/users/99df32ef8b264a0c8be44562148ae99e/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:03:01.463083 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc210df4-8b78-4fe1-a630-3c7fec49283f None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:01.463544 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc210df4-8b78-4fe1-a630-3c7fec49283f None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:01.464124 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc210df4-8b78-4fe1-a630-3c7fec49283f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:01.501332 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-dc210df4-8b78-4fe1-a630-3c7fec49283f None None] MFA Rules not processed for user `99df32ef8b264a0c8be44562148ae99e`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:03:01.502601 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-dc210df4-8b78-4fe1-a630-3c7fec49283f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:03:01.575417 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1431/2845] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:03:01 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:03:01.586337 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11395d0-9f07-4d91-8c35-b17061ab1c58 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:01.588117 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e11395d0-9f07-4d91-8c35-b17061ab1c58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:03:01.669497 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11395d0-9f07-4d91-8c35-b17061ab1c58 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:01.674996 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9fb7c9-d73a-446c-842b-95d911fbdf8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:01.675633 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fb7c9-d73a-446c-842b-95d911fbdf8b service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:01.675911 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fb7c9-d73a-446c-842b-95d911fbdf8b service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:01.676182 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fb7c9-d73a-446c-842b-95d911fbdf8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:01.744679 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9fb7c9-d73a-446c-842b-95d911fbdf8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:01.746150 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9fb7c9-d73a-446c-842b-95d911fbdf8b service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:01.759090 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1415/2846] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:03:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:03:11.575569 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=36 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:03:11.751605 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=70,keystone:INSERT=5 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:03:12.729594 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc210df4-8b78-4fe1-a630-3c7fec49283f None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:12.742857 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc210df4-8b78-4fe1-a630-3c7fec49283f None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:12.747209 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa9aff-213f-4c13-a937-f2508ff64af9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:12.747844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa9aff-213f-4c13-a937-f2508ff64af9 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:12.748124 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa9aff-213f-4c13-a937-f2508ff64af9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:12.748402 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa9aff-213f-4c13-a937-f2508ff64af9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:12.759834 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa9aff-213f-4c13-a937-f2508ff64af9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:12.761293 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa9aff-213f-4c13-a937-f2508ff64af9 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:12.771553 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1432/2847] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:03:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:03:22.767074 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=6 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:03:43.610584 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9fb7c9-d73a-446c-842b-95d911fbdf8b service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:43.740183 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9fb7c9-d73a-446c-842b-95d911fbdf8b service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:43.748224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ddfd4-d3aa-4a99-9e6c-806b9436a4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:43.749225 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ddfd4-d3aa-4a99-9e6c-806b9436a4ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:43.749511 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ddfd4-d3aa-4a99-9e6c-806b9436a4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:43.749828 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ddfd4-d3aa-4a99-9e6c-806b9436a4ee admin admin] PATH_INFO: `/v3/users/8cb76fa34bd5477c9bedc295efc0e62a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:43.751691 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ddfd4-d3aa-4a99-9e6c-806b9436a4ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cb76fa34bd5477c9bedc295efc0e62a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:43.753829 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ddfd4-d3aa-4a99-9e6c-806b9436a4ee admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:43.835574 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-cc7ddfd4-d3aa-4a99-9e6c-806b9436a4ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cb76fa34bd5477c9bedc295efc0e62a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:43.845569 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-cc7ddfd4-d3aa-4a99-9e6c-806b9436a4ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cb76fa34bd5477c9bedc295efc0e62a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:43.857162 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-cc7ddfd4-d3aa-4a99-9e6c-806b9436a4ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cb76fa34bd5477c9bedc295efc0e62a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:43.858186 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-cc7ddfd4-d3aa-4a99-9e6c-806b9436a4ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cb76fa34bd5477c9bedc295efc0e62a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:43.859332 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-cc7ddfd4-d3aa-4a99-9e6c-806b9436a4ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cb76fa34bd5477c9bedc295efc0e62a'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:43.888636 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1416/2848] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:03:43 2022] DELETE /identity/v3/users/8cb76fa34bd5477c9bedc295efc0e62a => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:03:43.894584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32b137-dede-43bc-945e-06688737008e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:43.895332 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32b137-dede-43bc-945e-06688737008e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:43.895563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32b137-dede-43bc-945e-06688737008e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:43.933469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-9e32b137-dede-43bc-945e-06688737008e None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:03:43.934752 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-9e32b137-dede-43bc-945e-06688737008e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:03:44.015239 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1433/2849] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:03:44.028649 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ddfd4-d3aa-4a99-9e6c-806b9436a4ee admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:44.031044 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-cc7ddfd4-d3aa-4a99-9e6c-806b9436a4ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:03:44.144224 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ddfd4-d3aa-4a99-9e6c-806b9436a4ee admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:44.151167 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9505f-9753-407b-ba1d-412a55e0d181 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:44.152264 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9505f-9753-407b-ba1d-412a55e0d181 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:44.152766 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9505f-9753-407b-ba1d-412a55e0d181 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:44.153271 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9505f-9753-407b-ba1d-412a55e0d181 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:44.170850 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9505f-9753-407b-ba1d-412a55e0d181 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:44.173711 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9505f-9753-407b-ba1d-412a55e0d181 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:44.192611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1417/2850] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:03:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:03:44.600014 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32b137-dede-43bc-945e-06688737008e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:44.602372 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-9e32b137-dede-43bc-945e-06688737008e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:03:44.620464 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32b137-dede-43bc-945e-06688737008e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:44.626403 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34805d-947d-4bc1-b508-61e47e614196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:44.627135 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34805d-947d-4bc1-b508-61e47e614196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:44.627544 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34805d-947d-4bc1-b508-61e47e614196 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:44.627812 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be34805d-947d-4bc1-b508-61e47e614196 admin admin] PATH_INFO: `/v3/projects/0975ce62e1d64b39aec547ff4b211926` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:44.629126 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be34805d-947d-4bc1-b508-61e47e614196 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0975ce62e1d64b39aec547ff4b211926)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:44.631262 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be34805d-947d-4bc1-b508-61e47e614196 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:44.709052 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-be34805d-947d-4bc1-b508-61e47e614196 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0975ce62e1d64b39aec547ff4b211926', 'request_id': 'req-be34805d-947d-4bc1-b508-61e47e614196'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:44.721303 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-be34805d-947d-4bc1-b508-61e47e614196 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0975ce62e1d64b39aec547ff4b211926', 'request_id': 'req-be34805d-947d-4bc1-b508-61e47e614196'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:44.733935 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-be34805d-947d-4bc1-b508-61e47e614196 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0975ce62e1d64b39aec547ff4b211926', 'request_id': 'req-be34805d-947d-4bc1-b508-61e47e614196'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:44.749679 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-be34805d-947d-4bc1-b508-61e47e614196 admin admin] The token cache is being invalidate because project 0975ce62e1d64b39aec547ff4b211926 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:03:44.750006 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-be34805d-947d-4bc1-b508-61e47e614196 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:44.750839 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-be34805d-947d-4bc1-b508-61e47e614196 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:44.762880 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1434/2851] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:03:44 2022] DELETE /identity/v3/projects/0975ce62e1d64b39aec547ff4b211926 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:03:44.767434 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9505f-9753-407b-ba1d-412a55e0d181 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:44.780933 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9505f-9753-407b-ba1d-412a55e0d181 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:44.818460 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:03:44.891666 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af8398-34aa-49cf-a5d0-225121344e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:44.892495 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af8398-34aa-49cf-a5d0-225121344e8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:44.892772 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af8398-34aa-49cf-a5d0-225121344e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:44.893048 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af8398-34aa-49cf-a5d0-225121344e8f admin admin] PATH_INFO: `/v3/users/99df32ef8b264a0c8be44562148ae99e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:44.894611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af8398-34aa-49cf-a5d0-225121344e8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=99df32ef8b264a0c8be44562148ae99e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:44.896833 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af8398-34aa-49cf-a5d0-225121344e8f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:44.975239 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-96af8398-34aa-49cf-a5d0-225121344e8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99df32ef8b264a0c8be44562148ae99e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:44.986931 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-96af8398-34aa-49cf-a5d0-225121344e8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99df32ef8b264a0c8be44562148ae99e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:44.996476 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-96af8398-34aa-49cf-a5d0-225121344e8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99df32ef8b264a0c8be44562148ae99e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:44.997552 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-96af8398-34aa-49cf-a5d0-225121344e8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99df32ef8b264a0c8be44562148ae99e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:44.998339 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-96af8398-34aa-49cf-a5d0-225121344e8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99df32ef8b264a0c8be44562148ae99e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:45.023029 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1418/2852] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:03:44 2022] DELETE /identity/v3/users/99df32ef8b264a0c8be44562148ae99e => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:03:45.409372 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34805d-947d-4bc1-b508-61e47e614196 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:45.411602 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-be34805d-947d-4bc1-b508-61e47e614196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:03:45.435097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be34805d-947d-4bc1-b508-61e47e614196 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:45.518697 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4550f1-c7bc-45a9-8c4b-7701d2d08b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:45.519760 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4550f1-c7bc-45a9-8c4b-7701d2d08b4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:45.520239 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4550f1-c7bc-45a9-8c4b-7701d2d08b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:45.520715 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4550f1-c7bc-45a9-8c4b-7701d2d08b4c admin admin] PATH_INFO: `/v3/projects/bddc5761485d4b28abd0501385ac0711` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:45.522794 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4550f1-c7bc-45a9-8c4b-7701d2d08b4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bddc5761485d4b28abd0501385ac0711)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:45.526147 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4550f1-c7bc-45a9-8c4b-7701d2d08b4c admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:45.581826 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:03:45.610177 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-1f4550f1-c7bc-45a9-8c4b-7701d2d08b4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bddc5761485d4b28abd0501385ac0711', 'request_id': 'req-1f4550f1-c7bc-45a9-8c4b-7701d2d08b4c'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:45.623210 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-1f4550f1-c7bc-45a9-8c4b-7701d2d08b4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bddc5761485d4b28abd0501385ac0711', 'request_id': 'req-1f4550f1-c7bc-45a9-8c4b-7701d2d08b4c'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:45.636266 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-1f4550f1-c7bc-45a9-8c4b-7701d2d08b4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bddc5761485d4b28abd0501385ac0711', 'request_id': 'req-1f4550f1-c7bc-45a9-8c4b-7701d2d08b4c'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:45.653574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-1f4550f1-c7bc-45a9-8c4b-7701d2d08b4c admin admin] The token cache is being invalidate because project bddc5761485d4b28abd0501385ac0711 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:03:45.653909 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-1f4550f1-c7bc-45a9-8c4b-7701d2d08b4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:45.654661 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-1f4550f1-c7bc-45a9-8c4b-7701d2d08b4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:45.656395 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1435/2853] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:03:45 2022] DELETE /identity/v3/projects/bddc5761485d4b28abd0501385ac0711 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:03:45.662532 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f1447-f149-4590-9a6a-3656fa1acbea None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:45.662806 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f1447-f149-4590-9a6a-3656fa1acbea None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:45.663254 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f1447-f149-4590-9a6a-3656fa1acbea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:45.691899 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-129f1447-f149-4590-9a6a-3656fa1acbea None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:03:45.693761 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-129f1447-f149-4590-9a6a-3656fa1acbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:03:45.783088 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1419/2854] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:03:45.793696 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f632ad-e2f6-4ae8-9f1e-0e5d1ba29ef0 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:45.794192 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f632ad-e2f6-4ae8-9f1e-0e5d1ba29ef0 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:45.794718 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f632ad-e2f6-4ae8-9f1e-0e5d1ba29ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:45.818267 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-85f632ad-e2f6-4ae8-9f1e-0e5d1ba29ef0 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:03:45.819629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-85f632ad-e2f6-4ae8-9f1e-0e5d1ba29ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:03:45.828708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1436/2855] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:03:45.833776 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f1447-f149-4590-9a6a-3656fa1acbea None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:45.850828 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f1447-f149-4590-9a6a-3656fa1acbea None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:45.859249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29355845-abef-4363-b088-e5c271987292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:45.860034 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29355845-abef-4363-b088-e5c271987292 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:45.860432 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29355845-abef-4363-b088-e5c271987292 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:45.860883 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29355845-abef-4363-b088-e5c271987292 admin admin] PATH_INFO: `/v3/domains` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:45.862093 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29355845-abef-4363-b088-e5c271987292 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93173) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 09:03:45.863310 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29355845-abef-4363-b088-e5c271987292 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:45.865501 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29355845-abef-4363-b088-e5c271987292 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:45.888492 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1420/2856] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 09:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 09:03:45.894590 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f632ad-e2f6-4ae8-9f1e-0e5d1ba29ef0 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:45.907508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f632ad-e2f6-4ae8-9f1e-0e5d1ba29ef0 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:45.914531 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599c943-d816-4585-9fc2-134351f2a3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:45.915398 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599c943-d816-4585-9fc2-134351f2a3c7 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:45.915732 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599c943-d816-4585-9fc2-134351f2a3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:45.916174 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599c943-d816-4585-9fc2-134351f2a3c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:45.916970 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a599c943-d816-4585-9fc2-134351f2a3c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:45.919436 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a599c943-d816-4585-9fc2-134351f2a3c7 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:45.965969 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1437/2857] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:03:45 2022] POST /identity/v3/projects => generated 387 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:03:45.967951 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29355845-abef-4363-b088-e5c271987292 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:45.984622 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29355845-abef-4363-b088-e5c271987292 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:46.032631 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:03:46.079550 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66334d33-a71f-432d-b235-5699f4459de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:46.080206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66334d33-a71f-432d-b235-5699f4459de8 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:46.080488 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66334d33-a71f-432d-b235-5699f4459de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:46.080764 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66334d33-a71f-432d-b235-5699f4459de8 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:46.081423 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66334d33-a71f-432d-b235-5699f4459de8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:46.083522 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66334d33-a71f-432d-b235-5699f4459de8 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:46.115816 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1421/2858] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:03:45 2022] POST /identity/v3/users => generated 294 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:03:46.121539 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599c943-d816-4585-9fc2-134351f2a3c7 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:46.140612 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599c943-d816-4585-9fc2-134351f2a3c7 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:46.147533 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a7d79-dc79-4821-ae7f-1a7950490d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:46.148526 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a7d79-dc79-4821-ae7f-1a7950490d3d admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:46.149035 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a7d79-dc79-4821-ae7f-1a7950490d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:46.149467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a7d79-dc79-4821-ae7f-1a7950490d3d admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:46.150679 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a7d79-dc79-4821-ae7f-1a7950490d3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:46.153579 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a7d79-dc79-4821-ae7f-1a7950490d3d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:46.174163 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1438/2859] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:03:46.178625 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66334d33-a71f-432d-b235-5699f4459de8 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:46.193073 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66334d33-a71f-432d-b235-5699f4459de8 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:46.198630 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262761c3-bfa4-4713-b79b-dde6bb6e6d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:46.199288 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262761c3-bfa4-4713-b79b-dde6bb6e6d5f admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:46.199566 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262761c3-bfa4-4713-b79b-dde6bb6e6d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:46.199842 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262761c3-bfa4-4713-b79b-dde6bb6e6d5f admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:46.200502 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262761c3-bfa4-4713-b79b-dde6bb6e6d5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:46.202155 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262761c3-bfa4-4713-b79b-dde6bb6e6d5f admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:46.218744 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1422/2860] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 09:03:46.223438 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a7d79-dc79-4821-ae7f-1a7950490d3d admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:46.240855 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a7d79-dc79-4821-ae7f-1a7950490d3d admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:46.248228 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16516a-7c81-4019-85f9-d95a9cce4760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:46.249467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16516a-7c81-4019-85f9-d95a9cce4760 admin admin] REQUEST_METHOD: `PUT` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:46.249981 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16516a-7c81-4019-85f9-d95a9cce4760 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:46.250480 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16516a-7c81-4019-85f9-d95a9cce4760 admin admin] PATH_INFO: `/v3/projects/d9b508a5d6664bde97debb6fe9571830/users/41986e884c3c48198e59ec41ea4bd4ba/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:46.279338 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f16516a-7c81-4019-85f9-d95a9cce4760 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9b508a5d6664bde97debb6fe9571830, user_id=41986e884c3c48198e59ec41ea4bd4ba, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:46.284698 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f16516a-7c81-4019-85f9-d95a9cce4760 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:46.307917 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1439/2861] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:03:46 2022] PUT /identity/v3/projects/d9b508a5d6664bde97debb6fe9571830/users/41986e884c3c48198e59ec41ea4bd4ba/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:03:46.323762 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e748e-df69-4f99-a11d-0e05571fea39 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:46.324523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e748e-df69-4f99-a11d-0e05571fea39 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:46.324992 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e748e-df69-4f99-a11d-0e05571fea39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:46.379015 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-9f0e748e-df69-4f99-a11d-0e05571fea39 None None] MFA Rules not processed for user `41986e884c3c48198e59ec41ea4bd4ba`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:03:46.381335 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9f0e748e-df69-4f99-a11d-0e05571fea39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:03:46.484019 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1423/2862] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:03:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:03:46.491183 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16516a-7c81-4019-85f9-d95a9cce4760 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:46.605720 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16516a-7c81-4019-85f9-d95a9cce4760 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:46.613870 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2589ac0-afc9-4447-9f7f-2a6c0b345f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:46.614939 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2589ac0-afc9-4447-9f7f-2a6c0b345f09 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:46.615430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2589ac0-afc9-4447-9f7f-2a6c0b345f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:46.615899 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2589ac0-afc9-4447-9f7f-2a6c0b345f09 admin admin] PATH_INFO: `/v3/projects` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:46.617017 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2589ac0-afc9-4447-9f7f-2a6c0b345f09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:46.620796 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2589ac0-afc9-4447-9f7f-2a6c0b345f09 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:46.664620 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1440/2863] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:03:46 2022] POST /identity/v3/projects => generated 389 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:03:46.670143 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e748e-df69-4f99-a11d-0e05571fea39 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:46.682509 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e748e-df69-4f99-a11d-0e05571fea39 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:46.771824 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ae898-3ee8-4d3c-a075-67bb7196d744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:46.772728 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ae898-3ee8-4d3c-a075-67bb7196d744 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:46.773213 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ae898-3ee8-4d3c-a075-67bb7196d744 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:46.773698 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ae898-3ee8-4d3c-a075-67bb7196d744 admin admin] PATH_INFO: `/v3/users` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:46.774838 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ae898-3ee8-4d3c-a075-67bb7196d744 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:46.778528 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ae898-3ee8-4d3c-a075-67bb7196d744 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:46.809880 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1424/2864] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:03:46 2022] POST /identity/v3/users => generated 301 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:03:46.814708 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2589ac0-afc9-4447-9f7f-2a6c0b345f09 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:46.826270 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2589ac0-afc9-4447-9f7f-2a6c0b345f09 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:46.831587 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d5d1e-655f-4517-8009-210ed24f313c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:46.832217 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d5d1e-655f-4517-8009-210ed24f313c admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:46.832489 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d5d1e-655f-4517-8009-210ed24f313c admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:46.833031 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d5d1e-655f-4517-8009-210ed24f313c admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:46.833767 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d5d1e-655f-4517-8009-210ed24f313c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:46.835430 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d5d1e-655f-4517-8009-210ed24f313c admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:46.847470 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1441/2865] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:03:46.852376 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ae898-3ee8-4d3c-a075-67bb7196d744 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:46.864124 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ae898-3ee8-4d3c-a075-67bb7196d744 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:46.869996 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee155ce4-30b0-4abd-8edd-398b93daa377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:46.870689 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee155ce4-30b0-4abd-8edd-398b93daa377 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:46.871048 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee155ce4-30b0-4abd-8edd-398b93daa377 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:46.871317 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee155ce4-30b0-4abd-8edd-398b93daa377 admin admin] PATH_INFO: `/v3/projects/544d8bb94ecc4e05ba18ce6fd485ff9e/users/52a35fefa1e147c6aedb64711f8f07c6/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:46.889586 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee155ce4-30b0-4abd-8edd-398b93daa377 admin admin] RBAC: Authorizing `identity:create_grant(project_id=544d8bb94ecc4e05ba18ce6fd485ff9e, user_id=52a35fefa1e147c6aedb64711f8f07c6, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:46.892359 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee155ce4-30b0-4abd-8edd-398b93daa377 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:46.909724 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1425/2866] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:03:46 2022] PUT /identity/v3/projects/544d8bb94ecc4e05ba18ce6fd485ff9e/users/52a35fefa1e147c6aedb64711f8f07c6/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 09:03:46.914168 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d5d1e-655f-4517-8009-210ed24f313c admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:46.927016 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d5d1e-655f-4517-8009-210ed24f313c admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:46.952486 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:03:47.002117 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d61c7e-00d3-4c79-bcd9-ad73b4c8fbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:47.003217 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d61c7e-00d3-4c79-bcd9-ad73b4c8fbc4 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:47.003702 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d61c7e-00d3-4c79-bcd9-ad73b4c8fbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:47.004179 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d61c7e-00d3-4c79-bcd9-ad73b4c8fbc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:47.005434 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d61c7e-00d3-4c79-bcd9-ad73b4c8fbc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:47.007278 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d61c7e-00d3-4c79-bcd9-ad73b4c8fbc4 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:47.021022 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1442/2867] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:03:47.026099 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee155ce4-30b0-4abd-8edd-398b93daa377 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:47.036464 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:03:47.043323 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee155ce4-30b0-4abd-8edd-398b93daa377 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:47.053036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64635704-d6dd-4c8d-b934-aa47e5f5c6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:47.053780 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64635704-d6dd-4c8d-b934-aa47e5f5c6b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:47.054108 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64635704-d6dd-4c8d-b934-aa47e5f5c6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:47.055028 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64635704-d6dd-4c8d-b934-aa47e5f5c6b5 admin admin] PATH_INFO: `/v3/projects/544d8bb94ecc4e05ba18ce6fd485ff9e/users/52a35fefa1e147c6aedb64711f8f07c6/roles/18c3c8a821b647f7982a955880bd0a9a` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:47.057420 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64635704-d6dd-4c8d-b934-aa47e5f5c6b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=544d8bb94ecc4e05ba18ce6fd485ff9e, user_id=52a35fefa1e147c6aedb64711f8f07c6, role_id=18c3c8a821b647f7982a955880bd0a9a)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:47.060344 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64635704-d6dd-4c8d-b934-aa47e5f5c6b5 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:47.073235 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1426/2868] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:03:47 2022] PUT /identity/v3/projects/544d8bb94ecc4e05ba18ce6fd485ff9e/users/52a35fefa1e147c6aedb64711f8f07c6/roles/18c3c8a821b647f7982a955880bd0a9a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 09:03:47.084584 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6a56e-00da-42bf-b849-b7b0317d1fa7 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:47.084873 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6a56e-00da-42bf-b849-b7b0317d1fa7 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:47.085152 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6a56e-00da-42bf-b849-b7b0317d1fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:47.118183 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-17a6a56e-00da-42bf-b849-b7b0317d1fa7 None None] MFA Rules not processed for user `52a35fefa1e147c6aedb64711f8f07c6`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:03:47.119862 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-17a6a56e-00da-42bf-b849-b7b0317d1fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:03:47.225502 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1443/2869] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:03:47 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:03:47.236960 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64635704-d6dd-4c8d-b934-aa47e5f5c6b5 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:47.239370 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-64635704-d6dd-4c8d-b934-aa47e5f5c6b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:03:47.330627 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64635704-d6dd-4c8d-b934-aa47e5f5c6b5 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:47.336554 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619d8f3-da59-4109-bfa4-81c423ae7c55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:47.337562 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619d8f3-da59-4109-bfa4-81c423ae7c55 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:47.338036 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619d8f3-da59-4109-bfa4-81c423ae7c55 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:47.338501 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619d8f3-da59-4109-bfa4-81c423ae7c55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:47.354240 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619d8f3-da59-4109-bfa4-81c423ae7c55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:47.359831 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619d8f3-da59-4109-bfa4-81c423ae7c55 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:47.389861 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1427/2870] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:03:47 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:03:47.564999 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6a56e-00da-42bf-b849-b7b0317d1fa7 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:47.650461 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6a56e-00da-42bf-b849-b7b0317d1fa7 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:47.657621 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de66815-14a1-4d8c-b205-bec05f0ae740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:47.658375 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de66815-14a1-4d8c-b205-bec05f0ae740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:47.658734 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de66815-14a1-4d8c-b205-bec05f0ae740 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:47.659271 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de66815-14a1-4d8c-b205-bec05f0ae740 admin admin] PATH_INFO: `/v3/users/41986e884c3c48198e59ec41ea4bd4ba` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:47.660830 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de66815-14a1-4d8c-b205-bec05f0ae740 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41986e884c3c48198e59ec41ea4bd4ba)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:47.663367 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de66815-14a1-4d8c-b205-bec05f0ae740 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:47.727495 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9de66815-14a1-4d8c-b205-bec05f0ae740 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41986e884c3c48198e59ec41ea4bd4ba'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:47.730246 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:03:47.747933 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9de66815-14a1-4d8c-b205-bec05f0ae740 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41986e884c3c48198e59ec41ea4bd4ba'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:47.758816 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9de66815-14a1-4d8c-b205-bec05f0ae740 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41986e884c3c48198e59ec41ea4bd4ba'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:47.768468 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9de66815-14a1-4d8c-b205-bec05f0ae740 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41986e884c3c48198e59ec41ea4bd4ba'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:47.769231 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-9de66815-14a1-4d8c-b205-bec05f0ae740 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41986e884c3c48198e59ec41ea4bd4ba'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:47.778782 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1444/2871] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:03:47 2022] DELETE /identity/v3/users/41986e884c3c48198e59ec41ea4bd4ba => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:03:47.783967 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774dea0-fc94-4891-94dc-5982c0a9add2 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:47.784206 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774dea0-fc94-4891-94dc-5982c0a9add2 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:47.784479 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774dea0-fc94-4891-94dc-5982c0a9add2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:47.805260 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-1774dea0-fc94-4891-94dc-5982c0a9add2 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:03:47.806475 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1774dea0-fc94-4891-94dc-5982c0a9add2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:03:47.874793 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1428/2872] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:03:47.887388 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de66815-14a1-4d8c-b205-bec05f0ae740 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:47.889026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-9de66815-14a1-4d8c-b205-bec05f0ae740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:03:47.972169 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de66815-14a1-4d8c-b205-bec05f0ae740 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:47.977209 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa22f2-6057-489f-bdf5-435cb92e60af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:47.977965 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa22f2-6057-489f-bdf5-435cb92e60af service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:47.978288 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa22f2-6057-489f-bdf5-435cb92e60af service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:47.978565 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa22f2-6057-489f-bdf5-435cb92e60af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:47.994141 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aa22f2-6057-489f-bdf5-435cb92e60af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:47.995643 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aa22f2-6057-489f-bdf5-435cb92e60af service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:48.007564 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1445/2873] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:03:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:03:48.332869 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774dea0-fc94-4891-94dc-5982c0a9add2 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:48.334610 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-1774dea0-fc94-4891-94dc-5982c0a9add2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:03:48.352835 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774dea0-fc94-4891-94dc-5982c0a9add2 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:48.361512 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7820d68-8dd0-4e70-ac46-4f062e3030c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:48.362557 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7820d68-8dd0-4e70-ac46-4f062e3030c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:48.363092 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7820d68-8dd0-4e70-ac46-4f062e3030c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:48.363561 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7820d68-8dd0-4e70-ac46-4f062e3030c9 admin admin] PATH_INFO: `/v3/projects/d9b508a5d6664bde97debb6fe9571830` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:48.366104 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7820d68-8dd0-4e70-ac46-4f062e3030c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9b508a5d6664bde97debb6fe9571830)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:48.370827 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7820d68-8dd0-4e70-ac46-4f062e3030c9 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:48.451279 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d7820d68-8dd0-4e70-ac46-4f062e3030c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9b508a5d6664bde97debb6fe9571830', 'request_id': 'req-d7820d68-8dd0-4e70-ac46-4f062e3030c9'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:48.461975 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d7820d68-8dd0-4e70-ac46-4f062e3030c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9b508a5d6664bde97debb6fe9571830', 'request_id': 'req-d7820d68-8dd0-4e70-ac46-4f062e3030c9'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:48.470283 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d7820d68-8dd0-4e70-ac46-4f062e3030c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9b508a5d6664bde97debb6fe9571830', 'request_id': 'req-d7820d68-8dd0-4e70-ac46-4f062e3030c9'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:48.487558 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d7820d68-8dd0-4e70-ac46-4f062e3030c9 admin admin] The token cache is being invalidate because project d9b508a5d6664bde97debb6fe9571830 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:03:48.487999 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d7820d68-8dd0-4e70-ac46-4f062e3030c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:48.489030 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d7820d68-8dd0-4e70-ac46-4f062e3030c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:48.491367 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1429/2874] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:03:48 2022] DELETE /identity/v3/projects/d9b508a5d6664bde97debb6fe9571830 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:03:48.495897 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa22f2-6057-489f-bdf5-435cb92e60af service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:48.507790 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa22f2-6057-489f-bdf5-435cb92e60af service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:48.606417 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02dc605-3723-490c-a847-e7ea29c6936d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:48.607432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02dc605-3723-490c-a847-e7ea29c6936d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:48.607432 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02dc605-3723-490c-a847-e7ea29c6936d admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:48.607768 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02dc605-3723-490c-a847-e7ea29c6936d admin admin] PATH_INFO: `/v3/users/52a35fefa1e147c6aedb64711f8f07c6` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:48.608931 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02dc605-3723-490c-a847-e7ea29c6936d admin admin] RBAC: Authorizing `identity:delete_user(user_id=52a35fefa1e147c6aedb64711f8f07c6)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:48.610960 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02dc605-3723-490c-a847-e7ea29c6936d admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:48.672508 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f02dc605-3723-490c-a847-e7ea29c6936d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52a35fefa1e147c6aedb64711f8f07c6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:48.690769 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f02dc605-3723-490c-a847-e7ea29c6936d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52a35fefa1e147c6aedb64711f8f07c6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:48.697018 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:03:48.704436 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f02dc605-3723-490c-a847-e7ea29c6936d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52a35fefa1e147c6aedb64711f8f07c6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:48.717275 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f02dc605-3723-490c-a847-e7ea29c6936d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52a35fefa1e147c6aedb64711f8f07c6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:48.718119 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f02dc605-3723-490c-a847-e7ea29c6936d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52a35fefa1e147c6aedb64711f8f07c6'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:48.728138 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1446/2875] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:03:48 2022] DELETE /identity/v3/users/52a35fefa1e147c6aedb64711f8f07c6 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:03:49.101638 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7820d68-8dd0-4e70-ac46-4f062e3030c9 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:49.103364 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d7820d68-8dd0-4e70-ac46-4f062e3030c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:03:49.122467 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7820d68-8dd0-4e70-ac46-4f062e3030c9 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:49.145030 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:03:49.216338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab711d-3075-488b-85ef-070a01e2d43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:49.216889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab711d-3075-488b-85ef-070a01e2d43e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:49.217170 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab711d-3075-488b-85ef-070a01e2d43e admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:49.217441 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab711d-3075-488b-85ef-070a01e2d43e admin admin] PATH_INFO: `/v3/projects/544d8bb94ecc4e05ba18ce6fd485ff9e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:49.218713 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab711d-3075-488b-85ef-070a01e2d43e admin admin] RBAC: Authorizing `identity:delete_project(project_id=544d8bb94ecc4e05ba18ce6fd485ff9e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:49.220725 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab711d-3075-488b-85ef-070a01e2d43e admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:49.298477 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-42ab711d-3075-488b-85ef-070a01e2d43e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '544d8bb94ecc4e05ba18ce6fd485ff9e', 'request_id': 'req-42ab711d-3075-488b-85ef-070a01e2d43e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:49.348596 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-42ab711d-3075-488b-85ef-070a01e2d43e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '544d8bb94ecc4e05ba18ce6fd485ff9e', 'request_id': 'req-42ab711d-3075-488b-85ef-070a01e2d43e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:49.355914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-42ab711d-3075-488b-85ef-070a01e2d43e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '544d8bb94ecc4e05ba18ce6fd485ff9e', 'request_id': 'req-42ab711d-3075-488b-85ef-070a01e2d43e'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:49.377088 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-42ab711d-3075-488b-85ef-070a01e2d43e admin admin] The token cache is being invalidate because project 544d8bb94ecc4e05ba18ce6fd485ff9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:03:49.377725 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-42ab711d-3075-488b-85ef-070a01e2d43e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:49.384132 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-42ab711d-3075-488b-85ef-070a01e2d43e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:49.386491 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1430/2876] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:03:49 2022] DELETE /identity/v3/projects/544d8bb94ecc4e05ba18ce6fd485ff9e => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:03:49.391846 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea9093-72f1-457a-a1e9-8c96470650a0 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:49.392199 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea9093-72f1-457a-a1e9-8c96470650a0 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:49.392370 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea9093-72f1-457a-a1e9-8c96470650a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:49.423262 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-79ea9093-72f1-457a-a1e9-8c96470650a0 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:03:49.425094 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-79ea9093-72f1-457a-a1e9-8c96470650a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:03:49.501224 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1447/2877] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:03:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:03:49.510616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d0bc4-800c-4004-9d50-1c4391688158 None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:49.511022 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d0bc4-800c-4004-9d50-1c4391688158 None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:49.511464 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d0bc4-800c-4004-9d50-1c4391688158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:49.539492 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-596d0bc4-800c-4004-9d50-1c4391688158 None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:03:49.540756 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-596d0bc4-800c-4004-9d50-1c4391688158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:03:49.550875 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1431/2878] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:03:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:03:49.556889 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea9093-72f1-457a-a1e9-8c96470650a0 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:49.569779 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea9093-72f1-457a-a1e9-8c96470650a0 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:49.576345 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f1fc4-e7e5-4eea-a2df-f56d708b23ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:49.577012 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f1fc4-e7e5-4eea-a2df-f56d708b23ad admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:49.577306 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f1fc4-e7e5-4eea-a2df-f56d708b23ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:49.577570 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f1fc4-e7e5-4eea-a2df-f56d708b23ad admin admin] PATH_INFO: `/v3/domains` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:49.578251 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f1fc4-e7e5-4eea-a2df-f56d708b23ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93172) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 09:03:49.578616 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f1fc4-e7e5-4eea-a2df-f56d708b23ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:49.580679 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f1fc4-e7e5-4eea-a2df-f56d708b23ad admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:49.597054 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1448/2879] 10.222.0.71 () {58 vars in 1153 bytes} [Fri Aug 26 09:03:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 09:03:49.602991 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d0bc4-800c-4004-9d50-1c4391688158 None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:49.615814 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d0bc4-800c-4004-9d50-1c4391688158 None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:49.621913 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ac608-df87-47ab-a80e-a482f68d1519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:49.622727 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ac608-df87-47ab-a80e-a482f68d1519 admin admin] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:49.623187 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ac608-df87-47ab-a80e-a482f68d1519 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:49.623436 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ac608-df87-47ab-a80e-a482f68d1519 admin admin] PATH_INFO: `/v3/projects` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:49.624231 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ac608-df87-47ab-a80e-a482f68d1519 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:49.626833 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ac608-df87-47ab-a80e-a482f68d1519 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:49.670825 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1432/2880] 10.222.0.71 () {60 vars in 1153 bytes} [Fri Aug 26 09:03:49 2022] POST /identity/v3/projects => generated 369 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:03:49.677307 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f1fc4-e7e5-4eea-a2df-f56d708b23ad admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:49.690563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f1fc4-e7e5-4eea-a2df-f56d708b23ad admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:49.792478 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736fbfd5-4e24-4d68-a073-cdabadf54b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:49.793348 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fbfd5-4e24-4d68-a073-cdabadf54b11 admin admin] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:49.793630 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fbfd5-4e24-4d68-a073-cdabadf54b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:49.793906 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736fbfd5-4e24-4d68-a073-cdabadf54b11 admin admin] PATH_INFO: `/v3/users` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:49.795230 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736fbfd5-4e24-4d68-a073-cdabadf54b11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:49.797091 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736fbfd5-4e24-4d68-a073-cdabadf54b11 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:49.823682 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1449/2881] 10.222.0.71 () {60 vars in 1144 bytes} [Fri Aug 26 09:03:49 2022] POST /identity/v3/users => generated 285 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 09:03:49.829749 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ac608-df87-47ab-a80e-a482f68d1519 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:49.843853 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ac608-df87-47ab-a80e-a482f68d1519 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:49.848955 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5ce3c-74db-4e4a-8c26-81b58461cd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:49.849608 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5ce3c-74db-4e4a-8c26-81b58461cd17 admin admin] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:49.849887 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5ce3c-74db-4e4a-8c26-81b58461cd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:49.850167 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5ce3c-74db-4e4a-8c26-81b58461cd17 admin admin] PATH_INFO: `/v3/roles` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:49.850937 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a5ce3c-74db-4e4a-8c26-81b58461cd17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:49.852766 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a5ce3c-74db-4e4a-8c26-81b58461cd17 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:49.870228 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1433/2882] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:03:49.872222 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736fbfd5-4e24-4d68-a073-cdabadf54b11 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:49.885964 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736fbfd5-4e24-4d68-a073-cdabadf54b11 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:49.893900 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd872c9-1488-41d9-a0e0-4c8e5fbfcc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:49.893900 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd872c9-1488-41d9-a0e0-4c8e5fbfcc51 admin admin] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:49.893900 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd872c9-1488-41d9-a0e0-4c8e5fbfcc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:49.893900 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd872c9-1488-41d9-a0e0-4c8e5fbfcc51 admin admin] PATH_INFO: `/v3/roles` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:49.897829 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd872c9-1488-41d9-a0e0-4c8e5fbfcc51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:49.897829 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd872c9-1488-41d9-a0e0-4c8e5fbfcc51 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:49.912991 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1450/2883] 10.222.0.71 () {58 vars in 1122 bytes} [Fri Aug 26 09:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 09:03:49.920042 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5ce3c-74db-4e4a-8c26-81b58461cd17 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:49.934919 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5ce3c-74db-4e4a-8c26-81b58461cd17 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:49.943590 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a52c0-d890-417c-a76f-acd4dd1ddeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:49.944448 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a52c0-d890-417c-a76f-acd4dd1ddeb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:49.944831 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a52c0-d890-417c-a76f-acd4dd1ddeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:49.945219 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a52c0-d890-417c-a76f-acd4dd1ddeb2 admin admin] PATH_INFO: `/v3/projects/b2b9fc48e91642b5a1c85c9a63e2f5b2/users/321fff599e754ac89b8fb96669d81f94/roles/b7a48a25b0cd479d85fe5395d64c458e` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:49.967970 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a52c0-d890-417c-a76f-acd4dd1ddeb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2b9fc48e91642b5a1c85c9a63e2f5b2, user_id=321fff599e754ac89b8fb96669d81f94, role_id=b7a48a25b0cd479d85fe5395d64c458e)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:49.972084 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a52c0-d890-417c-a76f-acd4dd1ddeb2 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:49.993069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1434/2884] 10.222.0.71 () {60 vars in 1483 bytes} [Fri Aug 26 09:03:49 2022] PUT /identity/v3/projects/b2b9fc48e91642b5a1c85c9a63e2f5b2/users/321fff599e754ac89b8fb96669d81f94/roles/b7a48a25b0cd479d85fe5395d64c458e => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:03:50.007114 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489557f-7e3e-4929-afba-768cc8f7f6e6 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:50.007419 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489557f-7e3e-4929-afba-768cc8f7f6e6 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:50.007697 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489557f-7e3e-4929-afba-768cc8f7f6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:50.044465 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-e489557f-7e3e-4929-afba-768cc8f7f6e6 None None] MFA Rules not processed for user `321fff599e754ac89b8fb96669d81f94`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:03:50.045920 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e489557f-7e3e-4929-afba-768cc8f7f6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:03:50.106648 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:03:50.138641 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1451/2885] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:03:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:03:50.153249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a52c0-d890-417c-a76f-acd4dd1ddeb2 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:50.155649 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-a51a52c0-d890-417c-a76f-acd4dd1ddeb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:03:50.224409 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:03:50.272715 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a52c0-d890-417c-a76f-acd4dd1ddeb2 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:50.278605 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0181be96-b9d4-4834-92d5-a299ec107f20 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0d178c717a3e49319545e70cb58cbbed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:50.279524 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0181be96-b9d4-4834-92d5-a299ec107f20 service glance] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:50.279815 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0181be96-b9d4-4834-92d5-a299ec107f20 service glance] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:50.280247 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0181be96-b9d4-4834-92d5-a299ec107f20 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:50.295329 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0181be96-b9d4-4834-92d5-a299ec107f20 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:50.296847 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0181be96-b9d4-4834-92d5-a299ec107f20 service glance] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:50.307799 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1435/2886] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:03:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:03:54.137225 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e489557f-7e3e-4929-afba-768cc8f7f6e6 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:54.139493 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e489557f-7e3e-4929-afba-768cc8f7f6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:03:54.236227 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e489557f-7e3e-4929-afba-768cc8f7f6e6 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:54.241128 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8082c11-4cee-43d2-b1c8-2ab689594cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:54.241919 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8082c11-4cee-43d2-b1c8-2ab689594cb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:54.242131 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8082c11-4cee-43d2-b1c8-2ab689594cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:54.242288 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8082c11-4cee-43d2-b1c8-2ab689594cb5 admin admin] PATH_INFO: `/v3/users/d5e175245f2e48c79d5f82dbaf4f1cf4` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:54.243530 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8082c11-4cee-43d2-b1c8-2ab689594cb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5e175245f2e48c79d5f82dbaf4f1cf4)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:54.245476 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8082c11-4cee-43d2-b1c8-2ab689594cb5 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:54.324174 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f8082c11-4cee-43d2-b1c8-2ab689594cb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5e175245f2e48c79d5f82dbaf4f1cf4'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:54.339878 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f8082c11-4cee-43d2-b1c8-2ab689594cb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5e175245f2e48c79d5f82dbaf4f1cf4'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:54.351002 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f8082c11-4cee-43d2-b1c8-2ab689594cb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5e175245f2e48c79d5f82dbaf4f1cf4'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:54.369142 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f8082c11-4cee-43d2-b1c8-2ab689594cb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5e175245f2e48c79d5f82dbaf4f1cf4'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:54.370321 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-f8082c11-4cee-43d2-b1c8-2ab689594cb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5e175245f2e48c79d5f82dbaf4f1cf4'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:54.382063 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1452/2887] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:03:54 2022] DELETE /identity/v3/users/d5e175245f2e48c79d5f82dbaf4f1cf4 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:03:54.645067 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0181be96-b9d4-4834-92d5-a299ec107f20 service glance] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:03:54.665953 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0181be96-b9d4-4834-92d5-a299ec107f20 service glance] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:03:54.757486 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd74f7-8c2a-4e7a-a1be-98c2182b57a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:03:54.758285 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd74f7-8c2a-4e7a-a1be-98c2182b57a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:03:54.758477 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd74f7-8c2a-4e7a-a1be-98c2182b57a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:03:54.758743 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd74f7-8c2a-4e7a-a1be-98c2182b57a6 admin admin] PATH_INFO: `/v3/projects/eed42d5a00d44ae3af1d74159e20c616` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:03:54.760045 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd74f7-8c2a-4e7a-a1be-98c2182b57a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eed42d5a00d44ae3af1d74159e20c616)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:03:54.763068 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd74f7-8c2a-4e7a-a1be-98c2182b57a6 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:03:54.823399 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-8dcd74f7-8c2a-4e7a-a1be-98c2182b57a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eed42d5a00d44ae3af1d74159e20c616', 'request_id': 'req-8dcd74f7-8c2a-4e7a-a1be-98c2182b57a6'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:54.832618 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-8dcd74f7-8c2a-4e7a-a1be-98c2182b57a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eed42d5a00d44ae3af1d74159e20c616', 'request_id': 'req-8dcd74f7-8c2a-4e7a-a1be-98c2182b57a6'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:54.838959 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-8dcd74f7-8c2a-4e7a-a1be-98c2182b57a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eed42d5a00d44ae3af1d74159e20c616', 'request_id': 'req-8dcd74f7-8c2a-4e7a-a1be-98c2182b57a6'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:54.854388 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-8dcd74f7-8c2a-4e7a-a1be-98c2182b57a6 admin admin] The token cache is being invalidate because project eed42d5a00d44ae3af1d74159e20c616 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:03:54.855007 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-8dcd74f7-8c2a-4e7a-a1be-98c2182b57a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:54.855718 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-8dcd74f7-8c2a-4e7a-a1be-98c2182b57a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:03:54.860424 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1436/2888] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:03:54 2022] DELETE /identity/v3/projects/eed42d5a00d44ae3af1d74159e20c616 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:04:04.374237 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=61,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:04:04.848846 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=67,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:04:18.614574 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8082c11-4cee-43d2-b1c8-2ab689594cb5 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:04:18.616186 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-f8082c11-4cee-43d2-b1c8-2ab689594cb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:04:18.632912 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8082c11-4cee-43d2-b1c8-2ab689594cb5 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:04:18.729563 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea29189-6818-4f9c-b01c-bc39e305627b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:04:18.730308 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea29189-6818-4f9c-b01c-bc39e305627b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:04:18.730689 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea29189-6818-4f9c-b01c-bc39e305627b admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:04:18.731173 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea29189-6818-4f9c-b01c-bc39e305627b admin admin] PATH_INFO: `/v3/users/321fff599e754ac89b8fb96669d81f94` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:04:18.732554 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea29189-6818-4f9c-b01c-bc39e305627b admin admin] RBAC: Authorizing `identity:delete_user(user_id=321fff599e754ac89b8fb96669d81f94)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:04:18.734571 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea29189-6818-4f9c-b01c-bc39e305627b admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:04:18.802425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2ea29189-6818-4f9c-b01c-bc39e305627b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '321fff599e754ac89b8fb96669d81f94'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:04:18.820208 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2ea29189-6818-4f9c-b01c-bc39e305627b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '321fff599e754ac89b8fb96669d81f94'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:04:18.831716 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2ea29189-6818-4f9c-b01c-bc39e305627b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '321fff599e754ac89b8fb96669d81f94'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:04:18.843199 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2ea29189-6818-4f9c-b01c-bc39e305627b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '321fff599e754ac89b8fb96669d81f94'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:04:18.844097 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-2ea29189-6818-4f9c-b01c-bc39e305627b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '321fff599e754ac89b8fb96669d81f94'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:04:18.853309 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1453/2889] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:04:18 2022] DELETE /identity/v3/users/321fff599e754ac89b8fb96669d81f94 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:04:18.858069 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f3740-8a07-4e52-9742-2a04495c027f None None] REQUEST_METHOD: `POST` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:04:18.858609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f3740-8a07-4e52-9742-2a04495c027f None None] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:04:18.858609 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f3740-8a07-4e52-9742-2a04495c027f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:04:18.880080 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.auth.core [None req-743f3740-8a07-4e52-9742-2a04495c027f None None] MFA Rules not processed for user `668b82af2c1345cca572ef92fff86ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=93173) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:04:18.881465 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-743f3740-8a07-4e52-9742-2a04495c027f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:04:18.967504 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1437/2890] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:04:18.981026 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea29189-6818-4f9c-b01c-bc39e305627b admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:04:18.982760 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-2ea29189-6818-4f9c-b01c-bc39e305627b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:04:19.069121 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea29189-6818-4f9c-b01c-bc39e305627b admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:04:19.076068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d17a8d-91ed-4ba4-ae59-6a861ae9b639 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:04:19.077154 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d17a8d-91ed-4ba4-ae59-6a861ae9b639 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:04:19.077636 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d17a8d-91ed-4ba4-ae59-6a861ae9b639 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:04:19.078131 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d17a8d-91ed-4ba4-ae59-6a861ae9b639 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:04:19.093029 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d17a8d-91ed-4ba4-ae59-6a861ae9b639 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:04:19.094554 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d17a8d-91ed-4ba4-ae59-6a861ae9b639 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:04:19.106421 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1454/2891] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:04:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:04:19.445330 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f3740-8a07-4e52-9742-2a04495c027f None None] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:04:19.447574 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-743f3740-8a07-4e52-9742-2a04495c027f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:04:19.462050 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f3740-8a07-4e52-9742-2a04495c027f None None] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:04:19.472763 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb22be2-3a9c-42f1-a14a-df55ef00d0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:04:19.473338 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb22be2-3a9c-42f1-a14a-df55ef00d0ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:04:19.473684 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb22be2-3a9c-42f1-a14a-df55ef00d0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:04:19.474083 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb22be2-3a9c-42f1-a14a-df55ef00d0ba admin admin] PATH_INFO: `/v3/projects/b2b9fc48e91642b5a1c85c9a63e2f5b2` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:04:19.475668 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb22be2-3a9c-42f1-a14a-df55ef00d0ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2b9fc48e91642b5a1c85c9a63e2f5b2)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:04:19.477632 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb22be2-3a9c-42f1-a14a-df55ef00d0ba admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:04:19.543119 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3eb22be2-3a9c-42f1-a14a-df55ef00d0ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2b9fc48e91642b5a1c85c9a63e2f5b2', 'request_id': 'req-3eb22be2-3a9c-42f1-a14a-df55ef00d0ba'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:04:19.555523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3eb22be2-3a9c-42f1-a14a-df55ef00d0ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2b9fc48e91642b5a1c85c9a63e2f5b2', 'request_id': 'req-3eb22be2-3a9c-42f1-a14a-df55ef00d0ba'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:04:19.568958 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3eb22be2-3a9c-42f1-a14a-df55ef00d0ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2b9fc48e91642b5a1c85c9a63e2f5b2', 'request_id': 'req-3eb22be2-3a9c-42f1-a14a-df55ef00d0ba'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:04:19.586358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3eb22be2-3a9c-42f1-a14a-df55ef00d0ba admin admin] The token cache is being invalidate because project b2b9fc48e91642b5a1c85c9a63e2f5b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93173) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:04:19.586767 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3eb22be2-3a9c-42f1-a14a-df55ef00d0ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:04:19.587463 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-3eb22be2-3a9c-42f1-a14a-df55ef00d0ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:04:19.588998 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1438/2892] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:04:19 2022] DELETE /identity/v3/projects/b2b9fc48e91642b5a1c85c9a63e2f5b2 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:04:25.754001 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ff5f2-63a6-443f-b72e-dce80e2651d1 None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:04:25.754576 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ff5f2-63a6-443f-b72e-dce80e2651d1 None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:04:25.754576 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ff5f2-63a6-443f-b72e-dce80e2651d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:04:25.777144 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-ff2ff5f2-63a6-443f-b72e-dce80e2651d1 None None] MFA Rules not processed for user `34eaab912a91496daeedbaf63f5ddcbb`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:04:25.778454 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ff2ff5f2-63a6-443f-b72e-dce80e2651d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:04:25.798178 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:04:25.844363 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1455/2893] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:04:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:04:29.583175 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:04:35.837793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=15 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:04:44.894692 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb22be2-3a9c-42f1-a14a-df55ef00d0ba admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:04:44.897375 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-3eb22be2-3a9c-42f1-a14a-df55ef00d0ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:04:44.989042 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb22be2-3a9c-42f1-a14a-df55ef00d0ba admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:04:44.996590 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb269fd9-4604-4db7-89dc-2684067cb78f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:04:44.997482 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb269fd9-4604-4db7-89dc-2684067cb78f service nova] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:04:44.997901 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb269fd9-4604-4db7-89dc-2684067cb78f service nova] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:04:44.998251 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb269fd9-4604-4db7-89dc-2684067cb78f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:04:45.000826 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-fb269fd9-4604-4db7-89dc-2684067cb78f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:04:45.014728 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb269fd9-4604-4db7-89dc-2684067cb78f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:04:45.016389 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb269fd9-4604-4db7-89dc-2684067cb78f service nova] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:04:45.029358 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1439/2894] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:04:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:04:45.407197 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ff5f2-63a6-443f-b72e-dce80e2651d1 None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:04:45.409748 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-ff2ff5f2-63a6-443f-b72e-dce80e2651d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:04:45.520033 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ff5f2-63a6-443f-b72e-dce80e2651d1 None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:04:45.530220 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e88937-ea01-4efe-8e25-e6e6de6473bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf5e2b4f5e440a7942adea3d51cec75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:04:45.531991 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e88937-ea01-4efe-8e25-e6e6de6473bf service placement] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:04:45.532529 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e88937-ea01-4efe-8e25-e6e6de6473bf service placement] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:04:45.533068 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e88937-ea01-4efe-8e25-e6e6de6473bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:04:45.535527 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e1e88937-ea01-4efe-8e25-e6e6de6473bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:04:45.558948 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e88937-ea01-4efe-8e25-e6e6de6473bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:04:45.561469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e88937-ea01-4efe-8e25-e6e6de6473bf service placement] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:04:45.574284 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1456/2895] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:04:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:04:55.023208 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=34 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:04:55.569940 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=32 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:06:24.246699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb269fd9-4604-4db7-89dc-2684067cb78f service nova] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:06:24.248611 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-fb269fd9-4604-4db7-89dc-2684067cb78f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:06:24.315388 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb269fd9-4604-4db7-89dc-2684067cb78f service nova] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:06:24.319698 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14fca3e-432a-4399-a029-29a49193d7e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:06:24.320315 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14fca3e-432a-4399-a029-29a49193d7e9 service cinder] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:06:24.320602 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14fca3e-432a-4399-a029-29a49193d7e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:06:24.320873 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14fca3e-432a-4399-a029-29a49193d7e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:06:24.322779 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e14fca3e-432a-4399-a029-29a49193d7e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:06:24.336855 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14fca3e-432a-4399-a029-29a49193d7e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:06:24.338376 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14fca3e-432a-4399-a029-29a49193d7e9 service cinder] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:06:24.350088 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1440/2896] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:06:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:06:32.460250 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae78c62-d943-4202-8f17-caeeec2fbe6e None None] REQUEST_METHOD: `POST` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:06:32.460914 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae78c62-d943-4202-8f17-caeeec2fbe6e None None] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:06:32.461036 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae78c62-d943-4202-8f17-caeeec2fbe6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:06:32.497388 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.auth.core [None req-7ae78c62-d943-4202-8f17-caeeec2fbe6e None None] MFA Rules not processed for user `34eaab912a91496daeedbaf63f5ddcbb`. Rule list: `[]` (Enabled: `True`). {{(pid=93172) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 09:06:32.498655 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-7ae78c62-d943-4202-8f17-caeeec2fbe6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:06:32.506206 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1457/2897] 10.222.0.71 () {58 vars in 958 bytes} [Fri Aug 26 09:06:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 09:06:32.892013 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14fca3e-432a-4399-a029-29a49193d7e9 service cinder] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:06:32.893230 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-e14fca3e-432a-4399-a029-29a49193d7e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:06:32.975135 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14fca3e-432a-4399-a029-29a49193d7e9 service cinder] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:06:32.979054 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8663b639-ab16-4d26-a4ea-1697c80a4789 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:06:32.979685 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663b639-ab16-4d26-a4ea-1697c80a4789 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:06:32.979960 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663b639-ab16-4d26-a4ea-1697c80a4789 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:06:32.980229 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663b639-ab16-4d26-a4ea-1697c80a4789 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:06:32.982020 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8663b639-ab16-4d26-a4ea-1697c80a4789 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:06:32.993483 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8663b639-ab16-4d26-a4ea-1697c80a4789 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:06:32.994963 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8663b639-ab16-4d26-a4ea-1697c80a4789 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:06:33.004410 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1441/2898] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:06:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:06:33.297049 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae78c62-d943-4202-8f17-caeeec2fbe6e None None] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:06:33.307420 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae78c62-d943-4202-8f17-caeeec2fbe6e None None] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:06:33.311660 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1549a8-a671-4001-b242-9072ec7d68e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '997b4ffc01fc467ebbd1092787f217f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:06:33.312342 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1549a8-a671-4001-b242-9072ec7d68e0 service cinder] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:06:33.312661 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1549a8-a671-4001-b242-9072ec7d68e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:06:33.312974 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1549a8-a671-4001-b242-9072ec7d68e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:06:33.315027 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-5e1549a8-a671-4001-b242-9072ec7d68e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:06:33.333999 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1549a8-a671-4001-b242-9072ec7d68e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:06:33.337828 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1549a8-a671-4001-b242-9072ec7d68e0 service cinder] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:06:33.351242 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1458/2899] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:06:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:06:33.669284 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8663b639-ab16-4d26-a4ea-1697c80a4789 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:06:33.671384 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-8663b639-ab16-4d26-a4ea-1697c80a4789 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:06:33.687249 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8663b639-ab16-4d26-a4ea-1697c80a4789 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:06:33.691976 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b12fbf-a84e-4853-93b9-5713fca17910 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:06:33.692924 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b12fbf-a84e-4853-93b9-5713fca17910 service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:06:33.693203 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b12fbf-a84e-4853-93b9-5713fca17910 service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:06:33.693578 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b12fbf-a84e-4853-93b9-5713fca17910 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:06:33.695951 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-15b12fbf-a84e-4853-93b9-5713fca17910 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:06:33.717889 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b12fbf-a84e-4853-93b9-5713fca17910 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:06:33.719518 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b12fbf-a84e-4853-93b9-5713fca17910 service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:06:33.732699 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1442/2900] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:06:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 09:06:43.346807 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=9 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:06:43.726146 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=58 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:06:49.651446 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1549a8-a671-4001-b242-9072ec7d68e0 service cinder] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:06:49.662910 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1549a8-a671-4001-b242-9072ec7d68e0 service cinder] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:06:49.667467 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e23a0-4354-4d72-93e4-441966723019 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:06:49.668079 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e23a0-4354-4d72-93e4-441966723019 service neutron] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:06:49.668360 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e23a0-4354-4d72-93e4-441966723019 service neutron] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:06:49.668633 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e23a0-4354-4d72-93e4-441966723019 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:06:49.670543 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a71e23a0-4354-4d72-93e4-441966723019 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:06:49.738253 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e23a0-4354-4d72-93e4-441966723019 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:06:49.739868 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e23a0-4354-4d72-93e4-441966723019 service neutron] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:06:49.751117 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1459/2901] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:06:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:06:59.745818 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=30 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:07:44.097605 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b12fbf-a84e-4853-93b9-5713fca17910 service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:07:44.109377 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b12fbf-a84e-4853-93b9-5713fca17910 service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:07:44.113317 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073099d0-8ed0-4372-915e-6202d617d28b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:07:44.113914 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073099d0-8ed0-4372-915e-6202d617d28b service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:07:44.114191 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073099d0-8ed0-4372-915e-6202d617d28b service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:07:44.114461 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073099d0-8ed0-4372-915e-6202d617d28b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:07:44.116383 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-073099d0-8ed0-4372-915e-6202d617d28b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:07:44.127690 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073099d0-8ed0-4372-915e-6202d617d28b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:07:44.129141 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073099d0-8ed0-4372-915e-6202d617d28b service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:07:44.138636 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1443/2902] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:07:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:07:54.134055 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=6 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:08:39.211482 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e23a0-4354-4d72-93e4-441966723019 service neutron] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:08:39.213113 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-a71e23a0-4354-4d72-93e4-441966723019 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:08:39.231018 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e23a0-4354-4d72-93e4-441966723019 service neutron] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:08:39.240944 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1b32a-8304-4396-b01f-35fe288ff7c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f16d1768715a41648af3fe6afadd7477', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:08:39.242053 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1b32a-8304-4396-b01f-35fe288ff7c2 service nova] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:08:39.242525 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1b32a-8304-4396-b01f-35fe288ff7c2 service nova] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:08:39.243378 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1b32a-8304-4396-b01f-35fe288ff7c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:08:39.247945 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-41b1b32a-8304-4396-b01f-35fe288ff7c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:08:39.269193 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b1b32a-8304-4396-b01f-35fe288ff7c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:08:39.272244 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b1b32a-8304-4396-b01f-35fe288ff7c2 service nova] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:08:39.284948 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1460/2903] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:08:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:08:42.778392 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073099d0-8ed0-4372-915e-6202d617d28b service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:08:42.789750 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073099d0-8ed0-4372-915e-6202d617d28b service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:08:42.794350 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f37bb-5631-414d-884d-d9561bea28ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4aa8938a504c4c4f8815478e520482d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:08:42.794969 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f37bb-5631-414d-884d-d9561bea28ae service neutron] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:08:42.795251 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f37bb-5631-414d-884d-d9561bea28ae service neutron] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:08:42.795523 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f37bb-5631-414d-884d-d9561bea28ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:08:42.797433 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9f3f37bb-5631-414d-884d-d9561bea28ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:08:42.868693 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f37bb-5631-414d-884d-d9561bea28ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:08:42.870268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f37bb-5631-414d-884d-d9561bea28ae service neutron] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:08:42.880467 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1444/2904] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:08:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:08:46.194425 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1b32a-8304-4396-b01f-35fe288ff7c2 service nova] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:08:46.206141 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1b32a-8304-4396-b01f-35fe288ff7c2 service nova] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:08:46.211622 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25783db-2927-4e53-91a5-92baa6fa58b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf5e2b4f5e440a7942adea3d51cec75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:08:46.212301 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25783db-2927-4e53-91a5-92baa6fa58b9 service placement] REQUEST_METHOD: `GET` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:08:46.212659 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25783db-2927-4e53-91a5-92baa6fa58b9 service placement] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:08:46.213021 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25783db-2927-4e53-91a5-92baa6fa58b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:08:46.214837 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-b25783db-2927-4e53-91a5-92baa6fa58b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:08:46.229963 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25783db-2927-4e53-91a5-92baa6fa58b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:08:46.231868 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25783db-2927-4e53-91a5-92baa6fa58b9 service placement] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:08:46.243327 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1461/2905] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:08:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:08:49.562115 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f37bb-5631-414d-884d-d9561bea28ae service neutron] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:08:49.564041 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-9f3f37bb-5631-414d-884d-d9561bea28ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:08:49.582926 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f37bb-5631-414d-884d-d9561bea28ae service neutron] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:08:49.589268 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4859f30-7057-4a6b-99ec-e2a41c62a149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:08:49.591567 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4859f30-7057-4a6b-99ec-e2a41c62a149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:08:49.591567 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4859f30-7057-4a6b-99ec-e2a41c62a149 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:08:49.591567 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4859f30-7057-4a6b-99ec-e2a41c62a149 admin admin] PATH_INFO: `/v3/users/34eaab912a91496daeedbaf63f5ddcbb` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:08:49.599193 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4859f30-7057-4a6b-99ec-e2a41c62a149 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34eaab912a91496daeedbaf63f5ddcbb)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:08:49.602179 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4859f30-7057-4a6b-99ec-e2a41c62a149 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:08:49.671626 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d4859f30-7057-4a6b-99ec-e2a41c62a149 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34eaab912a91496daeedbaf63f5ddcbb'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:08:49.684756 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d4859f30-7057-4a6b-99ec-e2a41c62a149 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34eaab912a91496daeedbaf63f5ddcbb'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:08:49.694102 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d4859f30-7057-4a6b-99ec-e2a41c62a149 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34eaab912a91496daeedbaf63f5ddcbb'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:08:49.694876 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d4859f30-7057-4a6b-99ec-e2a41c62a149 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34eaab912a91496daeedbaf63f5ddcbb'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:08:49.695527 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-d4859f30-7057-4a6b-99ec-e2a41c62a149 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34eaab912a91496daeedbaf63f5ddcbb'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:08:49.728105 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1445/2906] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:08:49 2022] DELETE /identity/v3/users/34eaab912a91496daeedbaf63f5ddcbb => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 09:08:49.946672 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25783db-2927-4e53-91a5-92baa6fa58b9 service placement] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:08:49.960799 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25783db-2927-4e53-91a5-92baa6fa58b9 service placement] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:08:50.031401 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c95b56-02b1-49e4-8954-9e79267c2bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:08:50.032183 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c95b56-02b1-49e4-8954-9e79267c2bb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:08:50.032469 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c95b56-02b1-49e4-8954-9e79267c2bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:08:50.032745 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c95b56-02b1-49e4-8954-9e79267c2bb1 admin admin] PATH_INFO: `/v3/projects/6773ba16da8845d39c90a627802cf05f` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:08:50.034046 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c95b56-02b1-49e4-8954-9e79267c2bb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6773ba16da8845d39c90a627802cf05f)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:08:50.035973 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c95b56-02b1-49e4-8954-9e79267c2bb1 admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:08:50.090045 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e0c95b56-02b1-49e4-8954-9e79267c2bb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6773ba16da8845d39c90a627802cf05f', 'request_id': 'req-e0c95b56-02b1-49e4-8954-9e79267c2bb1'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:08:50.096844 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e0c95b56-02b1-49e4-8954-9e79267c2bb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6773ba16da8845d39c90a627802cf05f', 'request_id': 'req-e0c95b56-02b1-49e4-8954-9e79267c2bb1'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:08:50.107371 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e0c95b56-02b1-49e4-8954-9e79267c2bb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6773ba16da8845d39c90a627802cf05f', 'request_id': 'req-e0c95b56-02b1-49e4-8954-9e79267c2bb1'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:08:50.119953 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e0c95b56-02b1-49e4-8954-9e79267c2bb1 admin admin] The token cache is being invalidate because project 6773ba16da8845d39c90a627802cf05f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:08:50.120265 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e0c95b56-02b1-49e4-8954-9e79267c2bb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:08:50.121020 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-e0c95b56-02b1-49e4-8954-9e79267c2bb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:08:50.122725 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1462/2907] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:08:49 2022] DELETE /identity/v3/projects/6773ba16da8845d39c90a627802cf05f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:08:50.127055 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4859f30-7057-4a6b-99ec-e2a41c62a149 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:08:50.128691 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-d4859f30-7057-4a6b-99ec-e2a41c62a149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:08:50.144631 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4859f30-7057-4a6b-99ec-e2a41c62a149 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:08:50.210046 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f243e9-594f-433c-9ba0-2d359647bb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:08:50.210723 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f243e9-594f-433c-9ba0-2d359647bb11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:08:50.211593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f243e9-594f-433c-9ba0-2d359647bb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:08:50.211593 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f243e9-594f-433c-9ba0-2d359647bb11 admin admin] PATH_INFO: `/v3/users/b03c1f7daa07427888a86c7e8a69dd4d` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:08:50.212551 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f243e9-594f-433c-9ba0-2d359647bb11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b03c1f7daa07427888a86c7e8a69dd4d)` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:08:50.214502 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f243e9-594f-433c-9ba0-2d359647bb11 admin admin] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:08:50.220649 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:08:50.268987 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-93f243e9-594f-433c-9ba0-2d359647bb11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b03c1f7daa07427888a86c7e8a69dd4d'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:08:50.279156 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-93f243e9-594f-433c-9ba0-2d359647bb11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b03c1f7daa07427888a86c7e8a69dd4d'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:08:50.288981 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-93f243e9-594f-433c-9ba0-2d359647bb11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b03c1f7daa07427888a86c7e8a69dd4d'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:08:50.289616 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-93f243e9-594f-433c-9ba0-2d359647bb11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b03c1f7daa07427888a86c7e8a69dd4d'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:08:50.290247 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.notifications [None req-93f243e9-594f-433c-9ba0-2d359647bb11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b03c1f7daa07427888a86c7e8a69dd4d'} {{(pid=93173) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:08:50.309930 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1446/2908] 10.222.0.71 () {58 vars in 1224 bytes} [Fri Aug 26 09:08:50 2022] DELETE /identity/v3/users/b03c1f7daa07427888a86c7e8a69dd4d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:08:50.519940 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c95b56-02b1-49e4-8954-9e79267c2bb1 admin admin] Authenticating user token {{(pid=93172) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:08:50.521902 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.fernet_utils [None req-e0c95b56-02b1-49e4-8954-9e79267c2bb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93172) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:08:50.536629 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c95b56-02b1-49e4-8954-9e79267c2bb1 admin admin] Validating token access rules against request {{(pid=93172) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:08:50.615310 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ff056-1527-4c0f-b975-b6eabcbaf42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '668b82af2c1345cca572ef92fff86ff2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '45132fb08c454ebaa1180c0b933c68b7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93172) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:08:50.615916 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ff056-1527-4c0f-b975-b6eabcbaf42a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:08:50.616194 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ff056-1527-4c0f-b975-b6eabcbaf42a admin admin] SCRIPT_NAME: `/identity` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:08:50.616466 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ff056-1527-4c0f-b975-b6eabcbaf42a admin admin] PATH_INFO: `/v3/projects/17ed1ac4468e42c5a8152cd5ce43eaad` {{(pid=93172) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:08:50.617606 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ff056-1527-4c0f-b975-b6eabcbaf42a admin admin] RBAC: Authorizing `identity:delete_project(project_id=17ed1ac4468e42c5a8152cd5ce43eaad)` {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:08:50.619598 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ff056-1527-4c0f-b975-b6eabcbaf42a admin admin] RBAC: Authorization granted {{(pid=93172) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:08:50.626538 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:08:50.673117 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-260ff056-1527-4c0f-b975-b6eabcbaf42a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17ed1ac4468e42c5a8152cd5ce43eaad', 'request_id': 'req-260ff056-1527-4c0f-b975-b6eabcbaf42a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:08:50.679668 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-260ff056-1527-4c0f-b975-b6eabcbaf42a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17ed1ac4468e42c5a8152cd5ce43eaad', 'request_id': 'req-260ff056-1527-4c0f-b975-b6eabcbaf42a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:08:50.687793 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-260ff056-1527-4c0f-b975-b6eabcbaf42a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17ed1ac4468e42c5a8152cd5ce43eaad', 'request_id': 'req-260ff056-1527-4c0f-b975-b6eabcbaf42a'} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:08:50.702956 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-260ff056-1527-4c0f-b975-b6eabcbaf42a admin admin] The token cache is being invalidate because project 17ed1ac4468e42c5a8152cd5ce43eaad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93172) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 09:08:50.703401 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-260ff056-1527-4c0f-b975-b6eabcbaf42a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:08:50.704444 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG keystone.notifications [None req-260ff056-1527-4c0f-b975-b6eabcbaf42a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93172) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 09:08:50.707011 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: [pid: 93172|app: 0|req: 1463/2909] 10.222.0.71 () {58 vars in 1233 bytes} [Fri Aug 26 09:08:50 2022] DELETE /identity/v3/projects/17ed1ac4468e42c5a8152cd5ce43eaad => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 09:09:00.306089 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:09:00.697000 dv-cinder-852364-5-iscsi devstack@keystone.service[93172]: DEBUG dbcounter [-] [93172] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93172) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 09:09:49.333218 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f243e9-594f-433c-9ba0-2d359647bb11 admin admin] Authenticating user token {{(pid=93173) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 09:09:49.334907 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-93f243e9-594f-433c-9ba0-2d359647bb11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:09:49.496046 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f243e9-594f-433c-9ba0-2d359647bb11 admin admin] Validating token access rules against request {{(pid=93173) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 09:09:49.564014 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76de072-1116-4500-8a69-cf08ef3b8c6b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cf5e2b4f5e440a7942adea3d51cec75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '543d44293d894a7fbb83401f7a26dd9b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93173) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 09:09:49.564640 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76de072-1116-4500-8a69-cf08ef3b8c6b service placement] REQUEST_METHOD: `GET` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 09:09:49.564913 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76de072-1116-4500-8a69-cf08ef3b8c6b service placement] SCRIPT_NAME: `/identity` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 09:09:49.565178 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76de072-1116-4500-8a69-cf08ef3b8c6b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93173) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 09:09:49.567878 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.fernet_utils [None req-f76de072-1116-4500-8a69-cf08ef3b8c6b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93173) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 09:09:49.580684 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76de072-1116-4500-8a69-cf08ef3b8c6b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 09:09:49.582207 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76de072-1116-4500-8a69-cf08ef3b8c6b service placement] RBAC: Authorization granted {{(pid=93173) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 09:09:49.593789 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: [pid: 93173|app: 0|req: 1447/2910] 10.222.0.71 () {60 vars in 1365 bytes} [Fri Aug 26 09:09:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 09:09:59.588465 dv-cinder-852364-5-iscsi devstack@keystone.service[93173]: DEBUG dbcounter [-] [93173] Writing DB stats keystone:SELECT=89 {{(pid=93173) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}